15:23:28 - NMC is in state NmcState.Ready and knows of 0 drawer(s) 15:23:28 - Found 0 PPS errors. 15:23:31 - Starting task "Configure modules" 15:23:31 - Configuring modules with configuration /local/home/cta_test_bench/config/2020/20200610/config_tower_53modules.NMC.xml 15:23:31 - Switching off high voltages in configuration 15:23:34 - Successfully configured 53 module(s) 15:23:35 - All front-end boards are programmed with firmware F55D2001 15:23:36 - All backplanes are programmed with firmware 15 15:23:41 - NMC is in state NmcState.Configured and knows of 53 drawer(s) 15:23:42 - Found 0 PPS errors. 15:23:42 - Finished task "Configure modules" 15:24:07 - Starting task "Switch HVPA voltage state" 15:24:07 - Setting HVPA current limit to 12489.75 uA 15:24:25 - Setting HVPA ramp speed to 25 V/s 15:24:27 - Switching HVPA to state Nominal 15:24:34 - High voltage locked channels will be checked every 30 s 15:24:36 - Setting HVPA current limits... 15:24:59 - Finished task "Switch HVPA voltage state" 15:25:03 - Checking for high voltage locked pixels... 15:25:03 - Module 76 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:25:03 - Module 77 has 2 pixel(s) locked (HVPA status of 0x79)! 15:25:03 - Module 78 has 1 pixel(s) locked (HVPA status of 0x7e)! 15:25:03 - Module 79 has 2 pixel(s) locked (HVPA status of 0x3d)! 15:25:04 - Module 85 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:25:04 - Module 94 has 1 pixel(s) locked (HVPA status of 0x7d)! 15:25:04 - Module 97 has 2 pixel(s) locked (HVPA status of 0x5b)! 15:25:04 - Module 101 has 2 pixel(s) locked (HVPA status of 0x5e)! 15:25:04 - Module 110 has 1 pixel(s) locked (HVPA status of 0x7e)! 15:25:04 - Module 112 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:25:04 - Module 113 has 2 pixel(s) locked (HVPA status of 0x2f)! 15:25:04 - Module 114 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:25:04 - Module 115 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:25:04 - Module 116 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:25:04 - Module 117 has 1 pixel(s) locked (HVPA status of 0x77)! 15:25:04 - Module 118 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:25:04 - Module 127 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:25:04 - Module 129 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:25:04 - Module 130 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:25:04 - Module 131 has 2 pixel(s) locked (HVPA status of 0x67)! 15:25:04 - Module 134 has 1 pixel(s) locked (HVPA status of 0x77)! 15:25:04 - Module 135 has 2 pixel(s) locked (HVPA status of 0x3e)! 15:25:04 - Module 146 has 2 pixel(s) locked (HVPA status of 0x3d)! 15:25:04 - Module 147 has 2 pixel(s) locked (HVPA status of 0x1f)! 15:25:04 - Module 149 has 3 pixel(s) locked (HVPA status of 0x4e)! 15:25:04 - Module 151 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:25:04 - Module 153 has 1 pixel(s) locked (HVPA status of 0x7e)! 15:25:04 - Module 154 has 2 pixel(s) locked (HVPA status of 0x75)! 15:25:04 - Module 163 has 1 pixel(s) locked (HVPA status of 0x77)! 15:25:05 - Module 164 has 2 pixel(s) locked (HVPA status of 0x76)! 15:25:05 - Module 166 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:25:05 - Module 167 has 2 pixel(s) locked (HVPA status of 0x37)! 15:25:05 - Module 170 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:25:05 - Unlocking 47 channel(s)... 15:25:05 - Resetting HVPA channel 2 of drawer 76... 15:25:05 - Resetting HVPA channel 1 of drawer 77... 15:25:05 - Resetting HVPA channel 2 of drawer 77... 15:25:06 - Resetting HVPA channel 0 of drawer 78... 15:25:06 - Resetting HVPA channel 1 of drawer 79... 15:25:06 - Resetting HVPA channel 6 of drawer 79... 15:25:06 - Resetting HVPA channel 6 of drawer 85... 15:25:06 - Resetting HVPA channel 1 of drawer 94... 15:25:07 - Resetting HVPA channel 2 of drawer 97... 15:25:07 - Resetting HVPA channel 5 of drawer 97... 15:25:07 - Resetting HVPA channel 0 of drawer 101... 15:25:07 - Resetting HVPA channel 5 of drawer 101... 15:25:08 - Resetting HVPA channel 0 of drawer 110... 15:25:08 - Resetting HVPA channel 4 of drawer 112... 15:25:08 - Resetting HVPA channel 4 of drawer 113... 15:25:08 - Resetting HVPA channel 6 of drawer 113... 15:25:09 - Resetting HVPA channel 6 of drawer 114... 15:25:09 - Resetting HVPA channel 2 of drawer 115... 15:25:09 - Resetting HVPA channel 5 of drawer 116... 15:25:09 - Resetting HVPA channel 3 of drawer 117... 15:25:10 - Resetting HVPA channel 4 of drawer 118... 15:25:10 - Resetting HVPA channel 2 of drawer 127... 15:25:10 - Resetting HVPA channel 5 of drawer 129... 15:25:10 - Resetting HVPA channel 2 of drawer 130... 15:25:10 - Resetting HVPA channel 3 of drawer 131... 15:25:11 - Resetting HVPA channel 4 of drawer 131... 15:25:11 - Resetting HVPA channel 3 of drawer 134... 15:25:11 - Resetting HVPA channel 0 of drawer 135... 15:25:11 - Resetting HVPA channel 6 of drawer 135... 15:25:12 - Resetting HVPA channel 1 of drawer 146... 15:25:12 - Resetting HVPA channel 6 of drawer 146... 15:25:12 - Resetting HVPA channel 5 of drawer 147... 15:25:12 - Resetting HVPA channel 6 of drawer 147... 15:25:13 - Resetting HVPA channel 0 of drawer 149... 15:25:13 - Resetting HVPA channel 4 of drawer 149... 15:25:13 - Resetting HVPA channel 5 of drawer 149... 15:25:13 - Resetting HVPA channel 2 of drawer 151... 15:25:14 - Resetting HVPA channel 0 of drawer 153... 15:25:14 - Resetting HVPA channel 1 of drawer 154... 15:25:14 - Resetting HVPA channel 3 of drawer 154... 15:25:14 - Resetting HVPA channel 3 of drawer 163... 15:25:14 - Resetting HVPA channel 0 of drawer 164... 15:25:15 - Resetting HVPA channel 3 of drawer 164... 15:25:15 - Resetting HVPA channel 5 of drawer 166... 15:25:15 - Resetting HVPA channel 3 of drawer 167... 15:25:15 - Resetting HVPA channel 6 of drawer 167... 15:25:16 - Resetting HVPA channel 6 of drawer 170... 15:25:22 - Calling method FSM.gotoObserving in object FSM 15:25:22 - Calling method FSM.gotoObserving in object FSM 15:25:22 - Failed to call FSM.gotoObserving: Bad statuses are present 15:25:33 - Checking for high voltage locked pixels... 15:25:33 - Module 78 has 1 pixel(s) locked (HVPA status of 0x7e)! 15:25:33 - Module 79 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:25:34 - Module 85 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:25:34 - Module 94 has 1 pixel(s) locked (HVPA status of 0x7d)! 15:25:34 - Module 97 has 2 pixel(s) locked (HVPA status of 0x5b)! 15:25:34 - Module 101 has 2 pixel(s) locked (HVPA status of 0x5e)! 15:25:34 - Module 112 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:25:34 - Module 113 has 2 pixel(s) locked (HVPA status of 0x2f)! 15:25:34 - Module 115 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:25:34 - Module 116 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:25:34 - Module 117 has 1 pixel(s) locked (HVPA status of 0x77)! 15:25:34 - Module 118 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:25:34 - Module 127 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:25:34 - Module 129 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:25:34 - Module 130 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:25:34 - Module 131 has 2 pixel(s) locked (HVPA status of 0x67)! 15:25:34 - Module 134 has 1 pixel(s) locked (HVPA status of 0x77)! 15:25:34 - Module 135 has 2 pixel(s) locked (HVPA status of 0x3e)! 15:25:34 - Module 146 has 2 pixel(s) locked (HVPA status of 0x3d)! 15:25:34 - Module 147 has 2 pixel(s) locked (HVPA status of 0x1f)! 15:25:34 - Module 149 has 2 pixel(s) locked (HVPA status of 0x4f)! 15:25:34 - Module 151 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:25:34 - Module 153 has 1 pixel(s) locked (HVPA status of 0x7e)! 15:25:34 - Module 154 has 2 pixel(s) locked (HVPA status of 0x75)! 15:25:34 - Module 163 has 1 pixel(s) locked (HVPA status of 0x77)! 15:25:35 - Module 164 has 2 pixel(s) locked (HVPA status of 0x76)! 15:25:35 - Module 166 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:25:35 - Module 167 has 2 pixel(s) locked (HVPA status of 0x37)! 15:25:35 - Module 170 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:25:35 - Unlocking 40 channel(s)... 15:25:35 - Resetting HVPA channel 0 of drawer 78... 15:25:35 - Resetting HVPA channel 6 of drawer 79... 15:25:35 - Resetting HVPA channel 6 of drawer 85... 15:25:35 - Resetting HVPA channel 1 of drawer 94... 15:25:36 - Resetting HVPA channel 2 of drawer 97... 15:25:36 - Resetting HVPA channel 5 of drawer 97... 15:25:36 - Resetting HVPA channel 0 of drawer 101... 15:25:36 - Resetting HVPA channel 5 of drawer 101... 15:25:37 - Resetting HVPA channel 4 of drawer 112... 15:25:37 - Resetting HVPA channel 4 of drawer 113... 15:25:37 - Resetting HVPA channel 6 of drawer 113... 15:25:37 - Resetting HVPA channel 2 of drawer 115... 15:25:38 - Resetting HVPA channel 5 of drawer 116... 15:25:38 - Resetting HVPA channel 3 of drawer 117... 15:25:38 - Resetting HVPA channel 4 of drawer 118... 15:25:38 - Resetting HVPA channel 2 of drawer 127... 15:25:39 - Resetting HVPA channel 5 of drawer 129... 15:25:39 - Resetting HVPA channel 2 of drawer 130... 15:25:39 - Resetting HVPA channel 3 of drawer 131... 15:25:39 - Resetting HVPA channel 4 of drawer 131... 15:25:39 - Resetting HVPA channel 3 of drawer 134... 15:25:40 - Resetting HVPA channel 0 of drawer 135... 15:25:40 - Resetting HVPA channel 6 of drawer 135... 15:25:40 - Resetting HVPA channel 1 of drawer 146... 15:25:40 - Resetting HVPA channel 6 of drawer 146... 15:25:41 - Resetting HVPA channel 5 of drawer 147... 15:25:41 - Resetting HVPA channel 6 of drawer 147... 15:25:41 - Resetting HVPA channel 4 of drawer 149... 15:25:41 - Resetting HVPA channel 5 of drawer 149... 15:25:42 - Resetting HVPA channel 2 of drawer 151... 15:25:42 - Resetting HVPA channel 0 of drawer 153... 15:25:42 - Resetting HVPA channel 1 of drawer 154... 15:25:42 - Resetting HVPA channel 3 of drawer 154... 15:25:42 - Resetting HVPA channel 3 of drawer 163... 15:25:43 - Resetting HVPA channel 0 of drawer 164... 15:25:43 - Resetting HVPA channel 3 of drawer 164... 15:25:43 - Resetting HVPA channel 5 of drawer 166... 15:25:43 - Resetting HVPA channel 3 of drawer 167... 15:25:44 - Resetting HVPA channel 6 of drawer 167... 15:25:44 - Resetting HVPA channel 6 of drawer 170... 15:25:55 - Calling method FSM.gotoStandby in object FSM 15:25:55 - Failed to call FSM.gotoStandby: Bad statuses are present 15:26:02 - Calling method FSM.gotoReady in object FSM 15:26:03 - Checking for high voltage locked pixels... 15:26:03 - Module 79 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:26:04 - Module 85 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:26:04 - Module 94 has 1 pixel(s) locked (HVPA status of 0x7d)! 15:26:04 - Module 97 has 2 pixel(s) locked (HVPA status of 0x5b)! 15:26:04 - Module 101 has 2 pixel(s) locked (HVPA status of 0x5e)! 15:26:04 - Module 112 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:26:04 - Module 113 has 2 pixel(s) locked (HVPA status of 0x2f)! 15:26:04 - Module 116 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:26:04 - Module 117 has 1 pixel(s) locked (HVPA status of 0x77)! 15:26:04 - Module 118 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:26:04 - Module 127 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:26:04 - Module 129 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:26:04 - Module 130 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:26:04 - Module 131 has 2 pixel(s) locked (HVPA status of 0x67)! 15:26:04 - Module 134 has 1 pixel(s) locked (HVPA status of 0x77)! 15:26:04 - Module 135 has 2 pixel(s) locked (HVPA status of 0x3e)! 15:26:04 - Module 146 has 2 pixel(s) locked (HVPA status of 0x3d)! 15:26:04 - Module 147 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:26:04 - Module 149 has 2 pixel(s) locked (HVPA status of 0x4f)! 15:26:04 - Module 151 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:26:04 - Module 153 has 1 pixel(s) locked (HVPA status of 0x7e)! 15:26:04 - Module 154 has 1 pixel(s) locked (HVPA status of 0x7d)! 15:26:04 - Module 163 has 1 pixel(s) locked (HVPA status of 0x77)! 15:26:05 - Module 164 has 2 pixel(s) locked (HVPA status of 0x76)! 15:26:05 - Module 166 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:26:05 - Module 167 has 1 pixel(s) locked (HVPA status of 0x77)! 15:26:05 - Module 170 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:26:05 - Unlocking 35 channel(s)... 15:26:05 - Resetting HVPA channel 6 of drawer 79... 15:26:05 - Resetting HVPA channel 6 of drawer 85... 15:26:05 - Resetting HVPA channel 1 of drawer 94... 15:26:06 - Resetting HVPA channel 2 of drawer 97... 15:26:06 - Resetting HVPA channel 5 of drawer 97... 15:26:06 - Resetting HVPA channel 0 of drawer 101... 15:26:06 - Resetting HVPA channel 5 of drawer 101... 15:26:06 - Resetting HVPA channel 4 of drawer 112... 15:26:07 - Resetting HVPA channel 4 of drawer 113... 15:26:07 - Resetting HVPA channel 6 of drawer 113... 15:26:07 - Resetting HVPA channel 5 of drawer 116... 15:26:07 - Resetting HVPA channel 3 of drawer 117... 15:26:08 - Resetting HVPA channel 4 of drawer 118... 15:26:08 - Resetting HVPA channel 2 of drawer 127... 15:26:08 - Resetting HVPA channel 5 of drawer 129... 15:26:08 - Resetting HVPA channel 2 of drawer 130... 15:26:09 - Resetting HVPA channel 3 of drawer 131... 15:26:09 - Resetting HVPA channel 4 of drawer 131... 15:26:09 - Resetting HVPA channel 3 of drawer 134... 15:26:09 - Resetting HVPA channel 0 of drawer 135... 15:26:10 - Resetting HVPA channel 6 of drawer 135... 15:26:10 - Resetting HVPA channel 1 of drawer 146... 15:26:10 - Resetting HVPA channel 6 of drawer 146... 15:26:10 - Resetting HVPA channel 5 of drawer 147... 15:26:10 - Resetting HVPA channel 4 of drawer 149... 15:26:11 - Resetting HVPA channel 5 of drawer 149... 15:26:11 - Resetting HVPA channel 2 of drawer 151... 15:26:11 - Resetting HVPA channel 0 of drawer 153... 15:26:11 - Resetting HVPA channel 1 of drawer 154... 15:26:12 - Resetting HVPA channel 3 of drawer 163... 15:26:12 - Resetting HVPA channel 0 of drawer 164... 15:26:12 - Resetting HVPA channel 3 of drawer 164... 15:26:12 - Resetting HVPA channel 5 of drawer 166... 15:26:13 - Resetting HVPA channel 3 of drawer 167... 15:26:13 - Resetting HVPA channel 6 of drawer 170... 15:26:18 - Calling method FSM.gotoObserving in object FSM 15:26:33 - Checking for high voltage locked pixels... 15:26:34 - Module 97 has 2 pixel(s) locked (HVPA status of 0x5b)! 15:26:34 - Module 101 has 1 pixel(s) locked (HVPA status of 0x7e)! 15:26:34 - Module 113 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:26:34 - Module 116 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:26:34 - Module 117 has 1 pixel(s) locked (HVPA status of 0x77)! 15:26:34 - Module 118 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:26:34 - Module 127 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:26:34 - Module 129 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:26:34 - Module 130 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:26:34 - Module 131 has 1 pixel(s) locked (HVPA status of 0x77)! 15:26:34 - Module 134 has 1 pixel(s) locked (HVPA status of 0x77)! 15:26:34 - Module 135 has 2 pixel(s) locked (HVPA status of 0x3e)! 15:26:34 - Module 146 has 2 pixel(s) locked (HVPA status of 0x3d)! 15:26:34 - Module 147 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:26:34 - Module 149 has 2 pixel(s) locked (HVPA status of 0x4f)! 15:26:34 - Module 151 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:26:34 - Module 153 has 1 pixel(s) locked (HVPA status of 0x7e)! 15:26:34 - Module 154 has 1 pixel(s) locked (HVPA status of 0x7d)! 15:26:34 - Module 163 has 1 pixel(s) locked (HVPA status of 0x77)! 15:26:35 - Module 164 has 2 pixel(s) locked (HVPA status of 0x76)! 15:26:35 - Module 166 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:26:35 - Module 167 has 1 pixel(s) locked (HVPA status of 0x77)! 15:26:35 - Unlocking 27 channel(s)... 15:26:35 - Resetting HVPA channel 2 of drawer 97... 15:26:35 - Resetting HVPA channel 5 of drawer 97... 15:26:35 - Resetting HVPA channel 0 of drawer 101... 15:26:35 - Resetting HVPA channel 6 of drawer 113... 15:26:36 - Resetting HVPA channel 5 of drawer 116... 15:26:36 - Resetting HVPA channel 3 of drawer 117... 15:26:36 - Resetting HVPA channel 4 of drawer 118... 15:26:36 - Resetting HVPA channel 2 of drawer 127... 15:26:37 - Resetting HVPA channel 5 of drawer 129... 15:26:37 - Resetting HVPA channel 2 of drawer 130... 15:26:37 - Resetting HVPA channel 3 of drawer 131... 15:26:37 - Resetting HVPA channel 3 of drawer 134... 15:26:38 - Resetting HVPA channel 0 of drawer 135... 15:26:38 - Resetting HVPA channel 6 of drawer 135... 15:26:38 - Resetting HVPA channel 1 of drawer 146... 15:26:38 - Resetting HVPA channel 6 of drawer 146... 15:26:39 - Resetting HVPA channel 5 of drawer 147... 15:26:39 - Resetting HVPA channel 4 of drawer 149... 15:26:39 - Resetting HVPA channel 5 of drawer 149... 15:26:39 - Resetting HVPA channel 2 of drawer 151... 15:26:39 - Resetting HVPA channel 0 of drawer 153... 15:26:40 - Resetting HVPA channel 1 of drawer 154... 15:26:40 - Resetting HVPA channel 3 of drawer 163... 15:26:40 - Resetting HVPA channel 0 of drawer 164... 15:26:40 - Resetting HVPA channel 3 of drawer 164... 15:26:41 - Resetting HVPA channel 5 of drawer 166... 15:26:41 - Resetting HVPA channel 3 of drawer 167... 15:26:50 - Starting NMC 15:26:50 - Clearing PPS error counters 15:27:03 - Checking for high voltage locked pixels... 15:27:04 - Module 97 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:27:04 - Module 101 has 1 pixel(s) locked (HVPA status of 0x7e)! 15:27:04 - Module 116 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:27:04 - Module 117 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:04 - Module 118 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:27:04 - Module 127 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:27:04 - Module 129 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:27:04 - Module 130 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:27:04 - Module 131 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:04 - Module 134 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:04 - Module 135 has 2 pixel(s) locked (HVPA status of 0x3e)! 15:27:04 - Module 146 has 2 pixel(s) locked (HVPA status of 0x3d)! 15:27:04 - Module 149 has 2 pixel(s) locked (HVPA status of 0x4f)! 15:27:04 - Module 154 has 1 pixel(s) locked (HVPA status of 0x7d)! 15:27:04 - Module 163 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:05 - Module 164 has 2 pixel(s) locked (HVPA status of 0x76)! 15:27:05 - Module 167 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:05 - Unlocking 21 channel(s)... 15:27:05 - Resetting HVPA channel 5 of drawer 97... 15:27:05 - Resetting HVPA channel 0 of drawer 101... 15:27:05 - Resetting HVPA channel 5 of drawer 116... 15:27:06 - Resetting HVPA channel 3 of drawer 117... 15:27:06 - Resetting HVPA channel 4 of drawer 118... 15:27:06 - Resetting HVPA channel 2 of drawer 127... 15:27:06 - Resetting HVPA channel 5 of drawer 129... 15:27:06 - Resetting HVPA channel 2 of drawer 130... 15:27:07 - Resetting HVPA channel 3 of drawer 131... 15:27:07 - Resetting HVPA channel 3 of drawer 134... 15:27:07 - Resetting HVPA channel 0 of drawer 135... 15:27:07 - Resetting HVPA channel 6 of drawer 135... 15:27:08 - Resetting HVPA channel 1 of drawer 146... 15:27:08 - Resetting HVPA channel 6 of drawer 146... 15:27:08 - Resetting HVPA channel 4 of drawer 149... 15:27:08 - Resetting HVPA channel 5 of drawer 149... 15:27:09 - Resetting HVPA channel 1 of drawer 154... 15:27:09 - Resetting HVPA channel 3 of drawer 163... 15:27:09 - Resetting HVPA channel 0 of drawer 164... 15:27:09 - Resetting HVPA channel 3 of drawer 164... 15:27:10 - Resetting HVPA channel 3 of drawer 167... 15:27:19 - Checking status of calibration box... 15:27:21 - Calibration box is configured with 1 leds at a frequency of 1000.0 Hz and a voltage of 10.00 V 15:27:23 - Stopping flashes... 15:27:33 - Checking for high voltage locked pixels... 15:27:34 - Module 97 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:27:34 - Module 117 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:34 - Module 118 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:27:34 - Module 130 has 1 pixel(s) locked (HVPA status of 0x7b)! 15:27:34 - Module 131 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:34 - Module 134 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:34 - Module 135 has 1 pixel(s) locked (HVPA status of 0x3f)! 15:27:34 - Module 146 has 2 pixel(s) locked (HVPA status of 0x3d)! 15:27:34 - Module 149 has 2 pixel(s) locked (HVPA status of 0x4f)! 15:27:34 - Module 154 has 1 pixel(s) locked (HVPA status of 0x7d)! 15:27:34 - Module 163 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:35 - Module 164 has 2 pixel(s) locked (HVPA status of 0x76)! 15:27:35 - Module 167 has 1 pixel(s) locked (HVPA status of 0x77)! 15:27:35 - Unlocking 16 channel(s)... 15:27:35 - Resetting HVPA channel 5 of drawer 97... 15:27:35 - Resetting HVPA channel 3 of drawer 117... 15:27:35 - Resetting HVPA channel 4 of drawer 118... 15:27:35 - Resetting HVPA channel 2 of drawer 130... 15:27:36 - Resetting HVPA channel 3 of drawer 131... 15:27:36 - Resetting HVPA channel 3 of drawer 134... 15:27:36 - Resetting HVPA channel 6 of drawer 135... 15:27:36 - Resetting HVPA channel 1 of drawer 146... 15:27:37 - Resetting HVPA channel 6 of drawer 146... 15:27:37 - Resetting HVPA channel 4 of drawer 149... 15:27:37 - Resetting HVPA channel 5 of drawer 149... 15:27:37 - Resetting HVPA channel 1 of drawer 154... 15:27:38 - Resetting HVPA channel 3 of drawer 163... 15:27:38 - Resetting HVPA channel 0 of drawer 164... 15:27:38 - Resetting HVPA channel 3 of drawer 164... 15:27:38 - Resetting HVPA channel 3 of drawer 167... 15:27:50 - Starting task "Check high voltages" 15:27:50 - Found 53 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state 15:27:58 - Actual voltage is between 106.14 V and 1167.54 V. 15:27:59 - Low voltages range between 300.00 V and 300.00 V. 15:28:03 - Checking for high voltage locked pixels... 15:28:04 - Module 97 has 1 pixel(s) locked (HVPA status of 0x5f)! 15:28:04 - Module 117 has 1 pixel(s) locked (HVPA status of 0x77)! 15:28:04 - Module 118 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:28:04 - Module 131 has 1 pixel(s) locked (HVPA status of 0x77)! 15:28:04 - Module 146 has 2 pixel(s) locked (HVPA status of 0x3d)! 15:28:04 - Module 149 has 1 pixel(s) locked (HVPA status of 0x6f)! 15:28:04 - Module 163 has 1 pixel(s) locked (HVPA status of 0x77)! 15:28:05 - Unlocking 8 channel(s)... 15:28:05 - Resetting HVPA channel 5 of drawer 97... 15:28:05 - Resetting HVPA channel 3 of drawer 117... 15:28:05 - Resetting HVPA channel 4 of drawer 118... 15:28:06 - Resetting HVPA channel 3 of drawer 131... 15:28:06 - Resetting HVPA channel 1 of drawer 146... 15:28:06 - Resetting HVPA channel 6 of drawer 146... 15:28:06 - Resetting HVPA channel 4 of drawer 149... 15:28:07 - Resetting HVPA channel 3 of drawer 163... 15:28:08 - Reference voltages range between 0.00 V and 1167.15 V. 15:28:17 - Maximum difference between expected and actual voltages is 956.73 V 15:28:22 - Maximum of current limits is 5647.07 uA 15:28:22 - Checking status of HVPA module in 53 drawer(s)... 15:28:22 - HVPA module in drawer 131 has a status of 0x77! 15:28:22 - HVPA module in drawer 146 has a status of 0x7d! 15:28:22 - HVPA module in drawer 149 has a status of 0x6f! 15:28:22 - HVPA module in drawer 163 has a status of 0x77! 15:28:22 - Checking status of HVPA channels in 53 drawer(s)... 15:28:23 - HVPA channel 1 of drawer 146 has status 0x8003! 15:28:25 - HVPA channel 3 of drawer 131 has status 0x8003! 15:28:25 - HVPA channel 3 of drawer 163 has status 0x8003! 15:28:25 - HVPA channel 4 of drawer 149 has status 0x8003! 15:28:26 - HV is locked in 4 channel(s), was in rise default in 0 channel(s) and was still in progress in 4 channel(s) 15:28:26 - Resetting 4 channel(s)... 15:28:26 - Resetting HVPA channel 1 of drawer 146... 15:28:27 - Resetting HVPA channel 3 of drawer 131... 15:28:28 - Resetting HVPA channel 3 of drawer 163... 15:28:29 - Resetting HVPA channel 4 of drawer 149... 15:28:33 - Checking for high voltage locked pixels... 15:28:40 - Checking reference and low voltages against /local/home/cta_test_bench/config/2020/20200610/config_tower_53modules.NMC.xml 15:28:45 - Done 15:28:45 - Finished task "Check high voltages" 15:29:02 - Starting task "Check high voltages" 15:29:02 - Found 53 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state 15:29:03 - Checking for high voltage locked pixels... 15:29:11 - Actual voltage is between 945.50 V and 1167.54 V. 15:29:11 - Low voltages range between 300.00 V and 300.00 V. 15:29:21 - Reference voltages range between 945.26 V and 1167.15 V. 15:29:29 - Maximum difference between expected and actual voltages is 0.71 V 15:29:33 - Checking for high voltage locked pixels... 15:29:34 - Maximum of current limits is 5647.07 uA 15:29:34 - Checking status of HVPA module in 53 drawer(s)... 15:29:35 - Checking status of HVPA channels in 53 drawer(s)... 15:29:39 - HV is stabilized in all channels. 15:29:39 - Checking reference and low voltages against /local/home/cta_test_bench/config/2020/20200610/config_tower_53modules.NMC.xml 15:29:44 - Done 15:29:44 - Finished task "Check high voltages" 15:30:03 - Checking for high voltage locked pixels... 15:30:11 - Status: TibState.PpsStopped 15:30:11 - Alarms: AlarmVector.ReadError 15:30:11 - Connection status: False TXclient_alive: False 15:30:11 - External delays: Local=0.000 ns, Calibration=65.000 ns, Sphe=150.000 ns 15:30:11 - Trigger mask: 0x74 = TriggerMask.SlowControl|Pedestal|Software|Calibration 15:30:11 - Input mask: 0x1 15:30:11 - Disable BUSY: False 15:30:11 - Extra BUSY Delay: 0.0 15:30:11 - Temperature: 34.31 15:30:14 - Configuring UCTS 15:30:14 - Configuring TIB... 15:30:14 - Setting TIB forced mode to False 15:30:14 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v 15:30:14 - Status: TibState.PpsStopped 15:30:14 - Alarms: AlarmVector.ReadError 15:30:14 - Connection status: False TXclient_alive: False 15:30:14 - Entering set_external_delay 15:30:14 - Setting ExternalDelayLocal to 0.000 ns 15:30:14 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v 15:30:15 - Entering set_external_delay 15:30:15 - Setting ExternalDelayCalibration to 65.000 ns 15:30:15 - Writing 65.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v 15:30:15 - Entering set_external_delay 15:30:15 - Setting ExternalDelaySphe to 150.000 ns 15:30:15 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v 15:30:16 - Setting TIB trigger mode to 0x71 = TriggerMask.SlowControl|Pedestal|Software|Mono 15:30:16 - Writing 113 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v 15:30:16 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v 15:30:16 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v 15:30:16 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v 15:30:16 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v 15:30:16 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v 15:30:16 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v 15:30:16 - Status: TibState.PpsStopped 15:30:16 - Alarms: AlarmVector.ReadError 15:30:16 - Connection status: False TXclient_alive: False 15:30:16 - Writing True to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v 15:30:16 - Writing 100 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v 15:30:16 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 15:30:16 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v 15:30:16 - Writing 100 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v 15:30:16 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v 15:30:16 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 15:30:16 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v 15:30:16 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v 15:30:16 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 15:30:23 - Resetting TIB... 15:30:30 - Configuring UCTS 15:30:30 - Configuring TIB... 15:30:30 - Setting TIB forced mode to False 15:30:30 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v 15:30:30 - Status: TibState.RunStopped 15:30:30 - Alarms: AlarmVector.NoAlarm 15:30:30 - Connection status: False TXclient_alive: False 15:30:30 - Disabling PPS 15:30:32 - Status: TibState.PpsStopped 15:30:32 - Alarms: AlarmVector.NoAlarm 15:30:32 - Connection status: False TXclient_alive: False 15:30:32 - Entering set_external_delay 15:30:32 - Setting ExternalDelayLocal to 0.000 ns 15:30:32 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v 15:30:32 - Entering set_external_delay 15:30:32 - Setting ExternalDelayCalibration to 65.000 ns 15:30:32 - Writing 65.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v 15:30:33 - Entering set_external_delay 15:30:33 - Setting ExternalDelaySphe to 150.000 ns 15:30:33 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v 15:30:33 - Setting TIB trigger mode to 0x74 = TriggerMask.SlowControl|Pedestal|Software|Calibration 15:30:33 - Writing 116 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v 15:30:33 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v 15:30:33 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v 15:30:33 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v 15:30:33 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v 15:30:33 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v 15:30:33 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v 15:30:33 - Status: TibState.PpsStopped 15:30:33 - Alarms: AlarmVector.NoAlarm 15:30:33 - Connection status: False TXclient_alive: False 15:30:33 - Writing True to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v 15:30:33 - Writing 100 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v 15:30:33 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 15:30:33 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v 15:30:33 - Writing 100 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v 15:30:33 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v 15:30:33 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 15:30:33 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v 15:30:33 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v 15:30:33 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 15:30:33 - Checking for high voltage locked pixels... 15:30:35 - Connecting TIB to camera server at 10.10.4.12:222 15:30:35 - Starting TIB... 15:30:35 - Status: TibState.PpsStopped 15:30:35 - Alarms: AlarmVector.NoAlarm 15:30:35 - Connection status: False TXclient_alive: False 15:30:35 - Resetting run 15:30:35 - Wait for UCTS external trigger before enabling trigger... 15:30:35 - Status: TibState.Standby 15:30:35 - Alarms: AlarmVector.NoAlarm 15:30:35 - Connection status: False TXclient_alive: True 15:30:35 - Status: TibState.Standby 15:30:35 - Alarms: AlarmVector.NoAlarm 15:30:35 - Connection status: False TXclient_alive: True 15:30:36 - Status: TibState.Armed 15:30:36 - Alarms: AlarmVector.NoAlarm 15:30:36 - Connection status: True TXclient_alive: True 15:30:42 - Configuring calibration box... 15:30:42 - Configuring calibration box with leds=0x40 voltage=10.00 frequency=1000.0/1 central_current=0.0 duration=300 width=1 lemo_out=True fiber1_out=True fiber2_out=False lemo_in=False enabled=False 15:30:44 - Starting TIB triggers... 15:30:44 - Enabling trigger 15:30:44 - Status: TibState.Running 15:30:44 - Alarms: AlarmVector.NoAlarm 15:30:44 - Connection status: True TXclient_alive: True 15:30:46 - Starting flashes... 15:31:03 - Checking for high voltage locked pixels... 15:31:16 - Stopping TIB... 15:31:16 - Status: TibState.Running 15:31:16 - Alarms: AlarmVector.ReadError 15:31:16 - Connection status: True TXclient_alive: True 15:31:16 - Disabling trigger 15:31:16 - Disabling PPS 15:31:17 - Status: TibState.PpsStopped 15:31:17 - Alarms: AlarmVector.ReadError 15:31:17 - Connection status: True TXclient_alive: True 15:31:17 - Closing TIB connection to camera server 15:31:22 - Status: TibState.PpsStopped 15:31:22 - Alarms: AlarmVector.ReadError 15:31:22 - Connection status: False TXclient_alive: False 15:31:22 - Run duration: 0:00:41 15:31:22 - Local camera triggers: 0 15:31:22 - Slow control triggers: 0 15:31:22 - Single photo-electron triggers: 0 15:31:22 - Calibration triggers: 29158 15:31:22 - Software triggers from UCTS: 0 15:31:22 - Pedestal triggers: 4015 15:31:22 - Collected input triggers: 32272 15:31:22 - Busy triggers: 48 15:31:22 - Camera event triggers sent to clusters: 32224