17:29:59 - ECC is in state EccState.Safe and transition EccTransition.Idle 17:29:59 - Active alarms: ['Alarm_General_current'] 17:29:59 - PDB temperature: 38.5 deg C 17:29:59 - Status of UPS power: V=232.0 V, I=0.80 A, P=0.090 kW, Q=4294.000 kVAR, S=0.180 kVA, PF=4294.00 17:29:59 - Status of 400 V: V1=233.0 V, V2=232.0 V, V3=233.0 V, I1=0.56 A, I2=2.15 A, I3=2.44 A, f=50.0 Hz, cos=0.53 17:29:59 - Status of 400 V: P1=0.070 kW, P2=0.490 kW, P3=4294.000 kW, Q1=0.080 kVAR, Q2=4294.000 kVAR, Q3=0.420 kVAR, S1=0.120 kVA, S2=0.510 kVA, S3=4294.000 kVA 17:29:59 - 0 relays enabled in PSB1. Total current of 0.0 A. 17:29:59 - 0 relays enabled in PSB2. Total current of -0.2 A. 17:29:59 - Status of PDB switches: StateBreaker=True, StateEmergencyStop=False, SwitchStatus_General400V=True, SwitchStatus_General230V=False, SwitchStatus_line400V=True, SwitchStatus_switch1=False, SwitchStatus_switch2=False, SwitchStatus_fanFront=True, SwitchStatus_fanRear=True, SwitchStatus_shutter=False 17:29:59 - Average air temperature: 23.8 deg C 17:29:59 - Air temperature (T1-T13): 23.8, 25.0, 27.6, 25.1, 24.5, 25.9, 24.5, 25.5, 24.2, 25.9, 25.4, 23.5, 21.7 deg C 17:29:59 - Air differential pressure: 0.2 Pa 17:29:59 - Back doors are closed. 17:29:59 - Light sensors: [1.5, 1.75] V 17:29:59 - Cooling valve is 100 %% open.Water flow: 9.8 l/min.Water pressure: [0.0, 0.0] bar 17:29:59 - Water temperature (T14-T16): 17.0, 17.0, 16.4 deg C 17:29:59 - Humidity: 50.2 %. 17:29:59 - Fan speeds: [300.0, 300.0, 300.0, 300.0, 300.0, 300.0] rpm 16:31:58 - Starting task "Prepare ECC" 16:31:58 - Preparing ECC... 16:31:58 - Setting ECC temperature range for Ready state to [16.000000, 36.000000] 16:31:58 - Setting ECC mode to EccTransition.Ready 16:31:59 - Received ECC state change notification: EccState.Safe 16:32:16 - Received ECC state change notification: EccState.Ready 16:32:17 - Setting power supply box SPE relay to False 16:32:18 - Setting power supply box MCB relay to False 16:32:18 - Finished task "Prepare ECC" 16:36:14 - Starting task "Powering modules..." 16:36:14 - Disabling power in all modules 16:36:14 - Finished task "Powering modules..." 16:36:17 - Starting task "Powering modules..." 16:36:17 - Enabling power in all modules 16:36:17 - Finished task "Powering modules..." 16:36:21 - Updating DTC module currents... 16:36:33 - Updating DTC module currents... 16:36:37 - Updating DTC module currents... 16:37:14 - Starting task "Configure modules" 16:37:14 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.6.0-41551 16:37:14 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.6.0-41551 16:37:14 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.6.0-41551 16:37:14 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.6.0-41551 16:37:14 - Configuring modules with configuration /data/config/2021/20210907/config_cameraqm_265febs_newnewDACL_newHV6.NMC.xml 16:37:14 - Switching off high voltages in configuration 16:37:19 - Could not invoke the MethodCall service: BadDisconnect 16:37:19 - No server with URI='urn:irfupca193.extra.cea.fr:UnifiedAutomation:UaServerCpp' has been discovered. The client could not discover a single server. 16:37:19 - Finished task "Configure modules" 16:37:24 - Starting task "Configure modules" 16:37:24 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.6.0-41551 16:37:24 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.6.0-41551 16:37:24 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.6.0-41551 16:37:24 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.6.0-41551 16:37:24 - Configuring modules with configuration /data/config/2021/20210907/config_cameraqm_265febs_newnewDACL_newHV6.NMC.xml 16:37:24 - Switching off high voltages in configuration 16:37:31 - ("NmcError: Failed to call 'Configure': status Bad statuses are present, code 0xfffffffe, status NmcStatus.Timeout", 4294967294) 16:37:31 - NMC is in state NmcState.Ready and knows of 0 drawer(s) 16:37:45 - Updating DTC module currents... 16:37:53 - Starting task "Configure modules" 16:37:53 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.6.0-41551 16:37:53 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.6.0-41551 16:37:53 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.6.0-41551 16:37:53 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.6.0-41551 16:37:53 - Configuring modules with configuration /data/config/2021/20210907/config_cameraqm_265febs_newnewDACL_newHV6.NMC.xml 16:37:53 - Switching off high voltages in configuration 16:38:00 - ("NmcError: Failed to call 'Configure': status Bad statuses are present, code 0xfffffffe, status NmcStatus.Timeout", 4294967294) 16:38:00 - NMC is in state NmcState.Ready and knows of 0 drawer(s) 16:38:08 - Updating DTC module currents... 16:38:11 - Starting task "Powering modules..." 16:38:11 - Disabling power in all modules 16:38:12 - Finished task "Powering modules..." 16:38:15 - Starting task "Powering modules..." 16:38:15 - Enabling power in all modules 16:38:15 - Finished task "Powering modules..." 16:38:18 - Updating DTC module currents... 16:38:20 - Module 127 connected to channel 8 of slot 9 has a CTDB current of 17.9 mA 16:38:20 - Module 132 connected to channel 13 of slot 9 has a CTDB current of 1.5 mA 16:38:20 - Module 134 connected to channel 15 of slot 9 has a CTDB current of 1.5 mA 16:38:20 - Module 205 connected to channel 12 of slot 5 has a CTDB current of 30.6 mA 16:38:24 - Scanning interface enp2s0 for non configured modules... 16:38:25 - Found 154 MAC address(es) 16:38:35 - Scanning interface enp2s0 for non configured modules... 16:38:36 - Found 220 MAC address(es) 16:38:37 - Scanning interface enp2s0 for non configured modules... 16:38:37 - Found 220 MAC address(es) 16:38:38 - Scanning interface enp2s0 for non configured modules... 16:38:38 - Found 220 MAC address(es) 16:38:39 - Scanning interface enp2s0 for non configured modules... 16:38:40 - Found 220 MAC address(es) 16:38:41 - Scanning interface enp2s0 for non configured modules... 16:38:41 - Found 220 MAC address(es) 16:38:42 - Scanning interface enp2s0 for non configured modules... 16:38:43 - Found 219 MAC address(es) 16:38:49 - Starting task "Configure modules" 16:38:49 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.6.0-41551 16:38:49 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.6.0-41551 16:38:49 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.6.0-41551 16:38:49 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.6.0-41551 16:38:49 - Configuring modules with configuration /data/config/2021/20210907/config_cameraqm_265febs_newnewDACL_newHV6.NMC.xml 16:38:49 - Switching off high voltages in configuration 16:38:59 - Only 261 module(s) out of 265 could be configured! 16:38:59 - Missing modules: [127, 132, 134, 205] 16:38:59 - Waiting for modules to be configured... 16:39:05 - All front-end boards are programmed with firmware F56D2002 16:39:13 - Found 2 different versions of DTBP firmware: [15 17] 16:39:13 - Module 127 is connected to channel 8 of DTC slot 9 16:39:16 - Starting task "Powering modules..." 16:39:16 - Disabling power in channel 8 of CTDB slot 9 16:39:16 - Finished task "Powering modules..." 16:39:17 - Starting task "Powering modules..." 16:39:17 - Enabling power in channel 8 of CTDB slot 9 16:39:18 - Finished task "Powering modules..." 16:39:20 - Module 134 is connected to channel 15 of DTC slot 9 16:39:22 - Starting task "Powering modules..." 16:39:22 - Disabling power in channel 15 of CTDB slot 9 16:39:22 - Finished task "Powering modules..." 16:39:23 - Starting task "Powering modules..." 16:39:23 - Enabling power in channel 15 of CTDB slot 9 16:39:23 - Finished task "Powering modules..." 16:39:27 - Module 132 is connected to channel 13 of DTC slot 9 16:39:29 - Starting task "Powering modules..." 16:39:29 - Disabling power in channel 13 of CTDB slot 9 16:39:30 - Finished task "Powering modules..." 16:39:32 - Starting task "Powering modules..." 16:39:32 - Enabling power in channel 13 of CTDB slot 9 16:39:32 - Finished task "Powering modules..." 16:39:36 - Module 205 is connected to channel 12 of DTC slot 5 16:39:38 - Starting task "Powering modules..." 16:39:38 - Disabling power in channel 12 of CTDB slot 5 16:39:39 - Finished task "Powering modules..." 16:39:40 - Starting task "Powering modules..." 16:39:40 - Enabling power in channel 12 of CTDB slot 5 16:39:41 - Finished task "Powering modules..." 16:39:44 - Updating DTC module currents... 16:39:46 - Module 134 connected to channel 15 of slot 9 has a CTDB current of 1.5 mA 16:39:52 - Module 134 is connected to channel 15 of DTC slot 9 16:39:52 - NMC is in state NmcState.Ready and knows of 261 drawer(s) 16:39:54 - Starting task "Powering modules..." 16:39:54 - Disabling power in channel 15 of CTDB slot 9 16:39:54 - Finished task "Powering modules..." 16:39:56 - Starting task "Powering modules..." 16:39:56 - Enabling power in channel 15 of CTDB slot 9 16:39:56 - Finished task "Powering modules..." 16:39:58 - Updating DTC module currents... 16:40:07 - All front-end boards have counted 0 event(s) 16:40:12 - Found PPS errors: 146 in drawer 0, 144 in drawer 1, 146 in drawer 2, 146 in drawer 3, 146 in drawer 4, 146 in drawer 5, 146 in drawer 8, 146 in drawer 9, 144 in drawer 10, 144 in drawer 11, 146 in drawer 12, 146 in drawer 13, 144 in drawer 14, 146 in drawer 15, 144 in drawer 16, 144 in drawer 17, 146 in drawer 18, 144 in drawer 19, 146 in drawer 20, 144 in drawer 21, 144 in drawer 22, 146 in drawer 23, 146 in drawer 24, 144 in drawer 25, 144 in drawer 26, 146 in drawer 27, 146 in drawer 28, 146 in drawer 29, 144 in drawer 30, 144 in drawer 31, 146 in drawer 32, 146 in drawer 33, 146 in drawer 34, 147 in drawer 35, 147 in drawer 36, 147 in drawer 37, 147 in drawer 38, 145 in drawer 39, 147 in drawer 40, 147 in drawer 41, 145 in drawer 42, 145 in drawer 43, 145 in drawer 44, 145 in drawer 45, 145 in drawer 46, 147 in drawer 47, 147 in drawer 48, 148 in drawer 49, 148 in drawer 50, 148 in drawer 51, 146 in drawer 52, 148 in drawer 53, 148 in drawer 54, 148 in drawer 55, 146 in drawer 56, 146 in drawer 57, 148 in drawer 58, 148 in drawer 59, 148 in drawer 60, 148 in drawer 61, 146 in drawer 62, 148 in drawer 63, 148 in drawer 64, 148 in drawer 65, 148 in drawer 66, 148 in drawer 67, 148 in drawer 68, 148 in drawer 69, 146 in drawer 70, 148 in drawer 71, 148 in drawer 72, 148 in drawer 73, 148 in drawer 74, 148 in drawer 75, 148 in drawer 76, 146 in drawer 77, 148 in drawer 78, 146 in drawer 79, 148 in drawer 80, 148 in drawer 81, 146 in drawer 82, 146 in drawer 83, 146 in drawer 84, 148 in drawer 85, 148 in drawer 86, 146 in drawer 87, 146 in drawer 88, 146 in drawer 89, 146 in drawer 90, 149 in drawer 91, 147 in drawer 92, 149 in drawer 93, 149 in drawer 94, 147 in drawer 95, 147 in drawer 96, 149 in drawer 97, 147 in drawer 98, 147 in drawer 99, 149 in drawer 100, 149 in drawer 101, 149 in drawer 102, 147 in drawer 103, 148 in drawer 104, 148 in drawer 105, 148 in drawer 106, 150 in drawer 107, 148 in drawer 108, 150 in drawer 109, 150 in drawer 110, 150 in drawer 111, 148 in drawer 112, 148 in drawer 113, 150 in drawer 114, 148 in drawer 115, 150 in drawer 116, 150 in drawer 117, 148 in drawer 118, 150 in drawer 119, 148 in drawer 120, 150 in drawer 121, 150 in drawer 122, 150 in drawer 123, 150 in drawer 124, 148 in drawer 125, 150 in drawer 126, 150 in drawer 128, 150 in drawer 129, 148 in drawer 130, 150 in drawer 131, 148 in drawer 133, 150 in drawer 135, 148 in drawer 136, 148 in drawer 137, 148 in drawer 138, 150 in drawer 139, 150 in drawer 140, 150 in drawer 141, 148 in drawer 142, 150 in drawer 143, 150 in drawer 144, 148 in drawer 145, 148 in drawer 146, 148 in drawer 147, 148 in drawer 148, 151 in drawer 149, 149 in drawer 150, 151 in drawer 151, 149 in drawer 152, 151 in drawer 153, 149 in drawer 154, 149 in drawer 155, 151 in drawer 156, 149 in drawer 157, 151 in drawer 158, 149 in drawer 159, 151 in drawer 160, 149 in drawer 161, 150 in drawer 162, 150 in drawer 163, 152 in drawer 164, 152 in drawer 165, 152 in drawer 166, 152 in drawer 167, 150 in drawer 168, 150 in drawer 169, 150 in drawer 170, 150 in drawer 171, 150 in drawer 172, 150 in drawer 173, 152 in drawer 174, 152 in drawer 175, 150 in drawer 176, 152 in drawer 177, 150 in drawer 178, 150 in drawer 179, 152 in drawer 180, 152 in drawer 181, 150 in drawer 182, 150 in drawer 183, 150 in drawer 184, 152 in drawer 185, 152 in drawer 186, 152 in drawer 187, 150 in drawer 188, 152 in drawer 189, 150 in drawer 190, 150 in drawer 191, 152 in drawer 192, 152 in drawer 193, 150 in drawer 194, 150 in drawer 195, 150 in drawer 196, 152 in drawer 197, 150 in drawer 198, 150 in drawer 199, 150 in drawer 200, 150 in drawer 201, 150 in drawer 202, 152 in drawer 203, 153 in drawer 204, 151 in drawer 206, 151 in drawer 207, 151 in drawer 208, 151 in drawer 209, 153 in drawer 210, 151 in drawer 211, 151 in drawer 212, 153 in drawer 213, 151 in drawer 214, 151 in drawer 215, 153 in drawer 216, 153 in drawer 217, 152 in drawer 218, 152 in drawer 219, 152 in drawer 220, 152 in drawer 221, 154 in drawer 222, 154 in drawer 223, 152 in drawer 224, 154 in drawer 225, 154 in drawer 226, 154 in drawer 227, 152 in drawer 228, 152 in drawer 229, 152 in drawer 230, 152 in drawer 231, 154 in drawer 232, 152 in drawer 233, 152 in drawer 234, 152 in drawer 235, 152 in drawer 236, 154 in drawer 237, 152 in drawer 238, 154 in drawer 239, 152 in drawer 240, 152 in drawer 241, 152 in drawer 242, 154 in drawer 243, 154 in drawer 244, 152 in drawer 245, 152 in drawer 246, 154 in drawer 247, 154 in drawer 248, 152 in drawer 249, 152 in drawer 250, 152 in drawer 251, 152 in drawer 252, 152 in drawer 253, 152 in drawer 254, 152 in drawer 255, 154 in drawer 256, 154 in drawer 257, 154 in drawer 258, 154 in drawer 259, 155 in drawer 260, 153 in drawer 261, 153 in drawer 262, 153 in drawer 263, 153 in drawer 264 16:40:12 - Finished task "Configure modules" 16:40:19 - Resetting NMC 16:40:25 - Starting task "Configure modules" 16:40:25 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.6.0-41551 16:40:25 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.6.0-41551 16:40:25 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.6.0-41551 16:40:25 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.6.0-41551 16:40:25 - Configuring modules with configuration /data/config/2021/20210907/config_cameraqm_265febs_newnewDACL_newHV6.NMC.xml 16:40:25 - Switching off high voltages in configuration 16:40:36 - Successfully configured 265 module(s) 16:40:36 - Waiting for modules to be configured... 16:40:42 - All front-end boards are programmed with firmware F56D2002 16:40:50 - Found 2 different versions of DTBP firmware: [15 17] 16:41:29 - Updating DTC module currents... 16:41:30 - NMC is in state NmcState.Ready and knows of 265 drawer(s) 16:41:44 - All front-end boards have counted 0 event(s) 16:41:50 - Found PPS errors: 148 in drawer 0, 148 in drawer 1, 148 in drawer 2, 148 in drawer 3, 148 in drawer 4, 148 in drawer 5, 148 in drawer 8, 148 in drawer 9, 148 in drawer 10, 148 in drawer 11, 146 in drawer 12, 146 in drawer 13, 146 in drawer 14, 146 in drawer 15, 146 in drawer 16, 146 in drawer 17, 146 in drawer 18, 148 in drawer 19, 148 in drawer 20, 148 in drawer 21, 148 in drawer 22, 148 in drawer 23, 148 in drawer 24, 148 in drawer 25, 148 in drawer 26, 148 in drawer 27, 148 in drawer 28, 146 in drawer 29, 146 in drawer 30, 148 in drawer 31, 148 in drawer 32, 148 in drawer 33, 148 in drawer 34, 148 in drawer 35, 148 in drawer 36, 148 in drawer 37, 148 in drawer 38, 148 in drawer 39, 149 in drawer 40, 149 in drawer 41, 149 in drawer 42, 149 in drawer 43, 149 in drawer 44, 149 in drawer 45, 149 in drawer 46, 149 in drawer 47, 149 in drawer 48, 149 in drawer 49, 149 in drawer 50, 147 in drawer 51, 149 in drawer 52, 148 in drawer 53, 150 in drawer 54, 150 in drawer 55, 150 in drawer 56, 150 in drawer 57, 150 in drawer 58, 150 in drawer 59, 150 in drawer 60, 150 in drawer 61, 150 in drawer 62, 150 in drawer 63, 148 in drawer 64, 148 in drawer 65, 148 in drawer 66, 148 in drawer 67, 150 in drawer 68, 150 in drawer 69, 148 in drawer 70, 148 in drawer 71, 150 in drawer 72, 150 in drawer 73, 150 in drawer 74, 148 in drawer 75, 150 in drawer 76, 150 in drawer 77, 150 in drawer 78, 150 in drawer 79, 148 in drawer 80, 148 in drawer 81, 150 in drawer 82, 148 in drawer 83, 150 in drawer 84, 150 in drawer 85, 148 in drawer 86, 150 in drawer 87, 150 in drawer 88, 150 in drawer 89, 150 in drawer 90, 150 in drawer 91, 148 in drawer 92, 150 in drawer 93, 149 in drawer 94, 151 in drawer 95, 149 in drawer 96, 149 in drawer 97, 149 in drawer 98, 151 in drawer 99, 149 in drawer 100, 151 in drawer 101, 151 in drawer 102, 151 in drawer 103, 151 in drawer 104, 149 in drawer 105, 149 in drawer 106, 150 in drawer 107, 150 in drawer 108, 152 in drawer 109, 152 in drawer 110, 152 in drawer 111, 152 in drawer 112, 152 in drawer 113, 152 in drawer 114, 152 in drawer 115, 152 in drawer 116, 152 in drawer 117, 152 in drawer 118, 152 in drawer 119, 152 in drawer 120, 150 in drawer 121, 152 in drawer 122, 150 in drawer 123, 152 in drawer 124, 150 in drawer 125, 150 in drawer 126, 152 in drawer 127, 152 in drawer 128, 150 in drawer 129, 152 in drawer 130, 152 in drawer 131, 152 in drawer 132, 152 in drawer 133, 152 in drawer 134, 152 in drawer 135, 152 in drawer 136, 152 in drawer 137, 150 in drawer 138, 150 in drawer 139, 152 in drawer 140, 152 in drawer 141, 152 in drawer 142, 152 in drawer 143, 152 in drawer 144, 152 in drawer 145, 152 in drawer 146, 153 in drawer 147, 153 in drawer 148, 153 in drawer 149, 153 in drawer 150, 153 in drawer 151, 153 in drawer 152, 151 in drawer 153, 153 in drawer 154, 151 in drawer 155, 153 in drawer 156, 153 in drawer 157, 153 in drawer 158, 153 in drawer 159, 153 in drawer 160, 154 in drawer 161, 152 in drawer 162, 154 in drawer 163, 154 in drawer 164, 154 in drawer 165, 154 in drawer 166, 154 in drawer 167, 154 in drawer 168, 154 in drawer 169, 154 in drawer 170, 154 in drawer 171, 154 in drawer 172, 152 in drawer 173, 152 in drawer 174, 152 in drawer 175, 154 in drawer 176, 154 in drawer 177, 154 in drawer 178, 154 in drawer 179, 154 in drawer 180, 152 in drawer 181, 154 in drawer 182, 154 in drawer 183, 152 in drawer 184, 154 in drawer 185, 154 in drawer 186, 154 in drawer 187, 154 in drawer 188, 154 in drawer 189, 152 in drawer 190, 154 in drawer 191, 154 in drawer 192, 154 in drawer 193, 152 in drawer 194, 154 in drawer 195, 152 in drawer 196, 154 in drawer 197, 152 in drawer 198, 154 in drawer 199, 154 in drawer 200, 155 in drawer 201, 153 in drawer 202, 155 in drawer 203, 153 in drawer 204, 153 in drawer 205, 153 in drawer 206, 153 in drawer 207, 155 in drawer 208, 155 in drawer 209, 155 in drawer 210, 155 in drawer 211, 155 in drawer 212, 155 in drawer 213, 156 in drawer 214, 156 in drawer 215, 154 in drawer 216, 156 in drawer 217, 154 in drawer 218, 154 in drawer 219, 154 in drawer 220, 156 in drawer 221, 156 in drawer 222, 156 in drawer 223, 156 in drawer 224, 154 in drawer 225, 156 in drawer 226, 154 in drawer 227, 156 in drawer 228, 156 in drawer 229, 154 in drawer 230, 156 in drawer 231, 154 in drawer 232, 156 in drawer 233, 156 in drawer 234, 156 in drawer 235, 156 in drawer 236, 156 in drawer 237, 156 in drawer 238, 156 in drawer 239, 154 in drawer 240, 154 in drawer 241, 156 in drawer 242, 156 in drawer 243, 156 in drawer 244, 156 in drawer 245, 154 in drawer 246, 156 in drawer 247, 156 in drawer 248, 156 in drawer 249, 156 in drawer 250, 156 in drawer 251, 154 in drawer 252, 154 in drawer 253, 156 in drawer 254, 155 in drawer 255, 155 in drawer 256, 157 in drawer 257, 155 in drawer 258, 157 in drawer 259, 155 in drawer 260, 157 in drawer 261, 157 in drawer 262, 157 in drawer 263, 155 in drawer 264 16:41:50 - Finished task "Configure modules" 16:42:00 - Updating DTC module currents... 16:42:08 - Updating FEB temperature widget... 17:11:56 - Updating DTC module currents... 17:13:03 - Updating FEB temperature widget... 17:13:21 - Updating voltage widget... 17:13:47 - Starting NMC 17:13:47 - Clearing PPS error counters 17:14:01 - Resetting FEB counters... 17:14:05 - Starting all configured modules... 17:14:24 - Checking UCTS clock... 17:14:24 - UCTS clock is late by 5 seconds! 17:14:24 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:14:24 - Error configuring UCTS: Failed to call Configure: Bad statuses are present 17:14:26 - Failed to call Reset: Bad statuses are present 17:14:55 - Checking UCTS clock... 17:14:56 - UCTS clock is late by 5 seconds! 17:14:56 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:14:56 - Error configuring UCTS: Failed to call Configure: Bad statuses are present 17:15:00 - Checking UCTS clock... 17:15:00 - UCTS clock is late by 5 seconds! 17:15:00 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:15:00 - Error configuring UCTS: Failed to call Configure: Bad statuses are present 17:15:01 - Checking UCTS clock... 17:15:02 - UCTS clock is late by 5 seconds! 17:15:02 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:15:02 - Error configuring UCTS: Failed to call Configure: Bad statuses are present 17:15:03 - Checking UCTS clock... 17:15:03 - UCTS clock is late by 5 seconds! 17:15:03 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:15:04 - Error configuring UCTS: Failed to call Configure: Bad statuses are present 17:15:06 - Checking UCTS clock... 17:15:07 - UCTS clock is late by 5 seconds! 17:15:07 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:15:07 - Error configuring UCTS: Failed to call Configure: Bad statuses are present 17:15:36 - Checking UCTS clock... 17:15:36 - UCTS clock is late by 1631204165 seconds! 17:15:37 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:15:37 - Configuring TIB... 17:15:37 - Setting TIB forced mode to False 17:15:37 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v 17:15:37 - Status: TibState.Undefined 17:15:37 - Alarms: AlarmVector.Undefined 17:15:37 - Connection status: False TXclient_alive: False 17:15:37 - Disabling PPS 17:15:37 - Error configuring TIB: Failed to call DisablePPS: Bad statuses are present 17:15:40 - Resetting TIB... 17:15:45 - Checking UCTS clock... 17:15:46 - UCTS clock is late by 1631204166 seconds! 17:15:46 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:15:46 - Configuring TIB... 17:15:46 - Setting TIB forced mode to False 17:15:46 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v 17:15:46 - Status: TibState.Undefined 17:15:46 - Alarms: AlarmVector.Undefined 17:15:46 - Connection status: False TXclient_alive: False 17:15:46 - Disabling PPS 17:15:46 - Error configuring TIB: Failed to call DisablePPS: Bad statuses are present 17:15:51 - Resetting TIB... 17:15:55 - Status: TibState.Undefined 17:15:55 - Alarms: AlarmVector.Undefined 17:15:55 - Connection status: False TXclient_alive: False 17:15:55 - External delays: Local=3825.000 ns, Calibration=3825.000 ns, Sphe=3825.000 ns 17:15:55 - Trigger mask: 0x51 = TriggerMask.SlowControl|Software|Mono 17:15:55 - Input mask: 0x7 17:15:55 - Disable BUSY: False 17:15:55 - Extra BUSY Delay: 0.0 17:15:55 - Temperature: 40.62 17:15:56 - Status: TibState.Undefined 17:15:56 - Alarms: AlarmVector.Undefined 17:15:56 - Connection status: False TXclient_alive: False 17:15:56 - External delays: Local=3825.000 ns, Calibration=3825.000 ns, Sphe=3825.000 ns 17:15:56 - Trigger mask: 0x51 = TriggerMask.SlowControl|Software|Mono 17:15:56 - Input mask: 0x7 17:15:56 - Disable BUSY: False 17:15:56 - Extra BUSY Delay: 0.0 17:15:56 - Temperature: 40.62 17:15:57 - Status: TibState.Undefined 17:15:57 - Alarms: AlarmVector.Undefined 17:15:57 - Connection status: False TXclient_alive: False 17:15:57 - External delays: Local=3825.000 ns, Calibration=3825.000 ns, Sphe=3825.000 ns 17:15:57 - Trigger mask: 0x51 = TriggerMask.SlowControl|Software|Mono 17:15:57 - Input mask: 0x7 17:15:57 - Disable BUSY: False 17:15:57 - Extra BUSY Delay: 0.0 17:15:57 - Temperature: 40.62 17:15:58 - Status: TibState.Undefined 17:15:58 - Alarms: AlarmVector.Undefined 17:15:58 - Connection status: False TXclient_alive: False 17:15:58 - External delays: Local=3825.000 ns, Calibration=3825.000 ns, Sphe=3825.000 ns 17:15:58 - Trigger mask: 0x51 = TriggerMask.SlowControl|Software|Mono 17:15:58 - Input mask: 0x7 17:15:58 - Disable BUSY: False 17:15:58 - Extra BUSY Delay: 0.0 17:15:58 - Temperature: 40.56 17:15:58 - Status: TibState.Undefined 17:15:58 - Alarms: AlarmVector.Undefined 17:15:58 - Connection status: False TXclient_alive: False 17:15:58 - External delays: Local=3825.000 ns, Calibration=3825.000 ns, Sphe=3825.000 ns 17:15:58 - Trigger mask: 0x51 = TriggerMask.SlowControl|Software|Mono 17:15:58 - Input mask: 0x7 17:15:59 - Disable BUSY: False 17:15:59 - Extra BUSY Delay: 0.0 17:15:59 - Temperature: 40.62 17:15:59 - Status: TibState.Undefined 17:15:59 - Alarms: AlarmVector.Undefined 17:15:59 - Connection status: False TXclient_alive: False 17:15:59 - External delays: Local=3825.000 ns, Calibration=3825.000 ns, Sphe=3825.000 ns 17:15:59 - Trigger mask: 0x51 = TriggerMask.SlowControl|Software|Mono 17:15:59 - Input mask: 0x7 17:15:59 - Disable BUSY: False 17:15:59 - Extra BUSY Delay: 0.0 17:15:59 - Temperature: 40.62 17:16:00 - Status: TibState.Undefined 17:16:00 - Alarms: AlarmVector.Undefined 17:16:00 - Connection status: False TXclient_alive: False 17:16:00 - External delays: Local=3825.000 ns, Calibration=3825.000 ns, Sphe=3825.000 ns 17:16:00 - Trigger mask: 0x51 = TriggerMask.SlowControl|Software|Mono 17:16:00 - Input mask: 0x7 17:16:00 - Disable BUSY: False 17:16:00 - Extra BUSY Delay: 0.0 17:16:00 - Temperature: 40.62 17:16:02 - Resetting TIB... 17:16:05 - Status: TibState.RunStopped 17:16:05 - Alarms: AlarmVector.NoAlarm 17:16:05 - Connection status: False TXclient_alive: False 17:16:05 - External delays: Local=3825.000 ns, Calibration=3825.000 ns, Sphe=3825.000 ns 17:16:05 - Trigger mask: 0x51 = TriggerMask.SlowControl|Software|Mono 17:16:05 - Input mask: 0x7 17:16:05 - Disable BUSY: False 17:16:05 - Extra BUSY Delay: 0.0 17:16:05 - Temperature: 40.62 17:16:09 - Checking UCTS clock... 17:16:10 - UCTS clock is late by 5 seconds! 17:16:10 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:16:10 - Configuring TIB... 17:16:10 - Setting TIB forced mode to False 17:16:10 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v 17:16:10 - Status: TibState.RunStopped 17:16:10 - Alarms: AlarmVector.NoAlarm 17:16:10 - Connection status: False TXclient_alive: False 17:16:10 - Disabling PPS 17:16:12 - Status: TibState.PpsStopped 17:16:12 - Alarms: AlarmVector.NoAlarm 17:16:12 - Connection status: False TXclient_alive: False 17:16:12 - Entering set_external_delay 17:16:12 - Setting ExternalDelayLocal to 27.000 ns 17:16:12 - Writing 27.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v 17:16:12 - Entering set_external_delay 17:16:12 - Setting ExternalDelayCalibration to 58.000 ns 17:16:12 - Writing 58.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v 17:16:12 - Entering set_external_delay 17:16:12 - Setting ExternalDelaySphe to 150.000 ns 17:16:12 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v 17:16:12 - Setting TIB trigger mode to 0x70 = TriggerMask.SlowControl|Pedestal|Software 17:16:12 - Writing 112 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v 17:16:12 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v 17:16:12 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v 17:16:12 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v 17:16:12 - Writing True to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v 17:16:12 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v 17:16:13 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v 17:16:13 - Status: TibState.PpsStopped 17:16:13 - Alarms: AlarmVector.NoAlarm 17:16:13 - Connection status: False TXclient_alive: False 17:16:13 - Writing True to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v 17:16:13 - Writing 1000 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v 17:16:13 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 17:16:13 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v 17:16:13 - Writing 1000 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v 17:16:13 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v 17:16:13 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 17:16:13 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v 17:16:13 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v 17:16:13 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 17:16:14 - Connecting TIB to camera server at 10.10.4.12:222 17:16:14 - Starting TIB... 17:16:14 - Status: TibState.PpsStopped 17:16:14 - Alarms: AlarmVector.NoAlarm 17:16:14 - Connection status: False TXclient_alive: False 17:16:14 - Resetting run 17:16:14 - Wait for UCTS external trigger before enabling trigger... 17:16:14 - Status: TibState.Standby 17:16:14 - Alarms: AlarmVector.NoAlarm 17:16:14 - Connection status: False TXclient_alive: True 17:16:14 - Status: TibState.Standby 17:16:14 - Alarms: AlarmVector.NoAlarm 17:16:14 - Connection status: False TXclient_alive: True 17:16:15 - Status: TibState.Armed 17:16:15 - Alarms: AlarmVector.NoAlarm 17:16:15 - Connection status: False TXclient_alive: True 17:16:16 - Starting TIB triggers... 17:16:16 - Enabling trigger 17:16:16 - Status: TibState.Running 17:16:16 - Alarms: AlarmVector.NoAlarm 17:16:16 - Connection status: False TXclient_alive: True 17:16:26 - Status: TibState.Running ConnectionStatus: False TXclient_alive: True 17:16:26 - Alarms: AlarmVector.BufferFull|AlmostFull 17:16:26 - Temperature: 40.56 17:16:26 - Local camera rate: 0.0 Hz 17:16:26 - Stereo trigger rate: 0.0 Hz 17:16:26 - SPE trigger rate: 0.0 Hz 17:16:26 - Calibration trigger rate: 0.0 Hz 17:16:26 - Software trigger rate: 0.0 Hz 17:16:26 - Pedestal trigger rate: 1000.0 Hz 17:16:26 - Collected trigger rate: 1000.0 Hz 17:16:26 - Busy rate: 0.0 Hz 17:16:26 - Camera trigger rate: 1000.0 Hz 17:16:31 - Stopping TIB... 17:16:31 - Status: TibState.Running 17:16:31 - Alarms: AlarmVector.BufferFull|AlmostFull 17:16:31 - Connection status: False TXclient_alive: True 17:16:31 - Disabling trigger 17:16:31 - Disabling PPS 17:16:32 - Status: TibState.PpsStopped 17:16:32 - Alarms: AlarmVector.BufferFull|AlmostFull 17:16:32 - Connection status: False TXclient_alive: True 17:16:33 - Closing TIB connection to camera server 17:16:40 - Calling method FSM.gotoReady in object FSM 17:16:40 - Failed to call FSM.gotoReady: Bad statuses are present 17:16:43 - Stopping NMC 17:16:49 - Found 0 PPS errors. 17:16:51 - Starting NMC 17:16:51 - Clearing PPS error counters 17:17:05 - Resetting FEB counters... 17:17:08 - Starting all configured modules... 17:17:18 - Calling method FSM.gotoObserving in object FSM 17:17:18 - Starting run 2580 17:17:30 - Checking UCTS clock... 17:17:30 - UCTS clock is late by 5 seconds! 17:17:30 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 17:17:30 - Configuring TIB... 17:17:30 - Setting TIB forced mode to False 17:17:30 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v 17:17:30 - Status: TibState.PpsStopped 17:17:30 - Alarms: AlarmVector.BufferFull|AlmostFull 17:17:30 - Connection status: False TXclient_alive: False 17:17:30 - Entering set_external_delay 17:17:30 - Setting ExternalDelayLocal to 27.000 ns 17:17:30 - Writing 27.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v 17:17:31 - Entering set_external_delay 17:17:31 - Setting ExternalDelayCalibration to 58.000 ns 17:17:31 - Writing 58.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v 17:17:31 - Entering set_external_delay 17:17:31 - Setting ExternalDelaySphe to 150.000 ns 17:17:31 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v 17:17:31 - Setting TIB trigger mode to 0x70 = TriggerMask.SlowControl|Pedestal|Software 17:17:31 - Writing 112 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v 17:17:31 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v 17:17:31 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v 17:17:31 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v 17:17:31 - Writing True to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v 17:17:31 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v 17:17:31 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v 17:17:31 - Status: TibState.PpsStopped 17:17:31 - Alarms: AlarmVector.BufferFull|AlmostFull 17:17:31 - Connection status: False TXclient_alive: False 17:17:31 - Writing True to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v 17:17:31 - Writing 1000 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v 17:17:31 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 17:17:31 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v 17:17:31 - Writing 1000 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v 17:17:31 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v 17:17:31 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 17:17:31 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v 17:17:31 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v 17:17:31 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 17:17:33 - Connecting TIB to camera server at 10.10.4.12:222 17:17:33 - Starting TIB... 17:17:33 - Status: TibState.PpsStopped 17:17:33 - Alarms: AlarmVector.BufferFull|AlmostFull 17:17:33 - Connection status: False TXclient_alive: True 17:17:33 - Resetting run 17:17:33 - Wait for UCTS external trigger before enabling trigger... 17:17:33 - Status: TibState.Standby 17:17:33 - Alarms: AlarmVector.NoAlarm 17:17:33 - Connection status: False TXclient_alive: True 17:17:33 - Status: TibState.Standby 17:17:33 - Alarms: AlarmVector.NoAlarm 17:17:33 - Connection status: False TXclient_alive: True 17:17:34 - Status: TibState.Armed 17:17:34 - Alarms: AlarmVector.NoAlarm 17:17:34 - Connection status: True TXclient_alive: True 17:17:36 - Starting TIB triggers... 17:17:36 - Enabling trigger 17:17:36 - Status: TibState.Running 17:17:36 - Alarms: AlarmVector.ReadError 17:17:36 - Connection status: True TXclient_alive: True 17:17:54 - Stopping TIB... 17:17:54 - Status: TibState.Running 17:17:54 - Alarms: AlarmVector.ReadError 17:17:54 - Connection status: True TXclient_alive: True 17:17:54 - Disabling trigger 17:17:55 - Disabling PPS 17:17:56 - Status: TibState.PpsStopped 17:17:56 - Alarms: AlarmVector.ReadError 17:17:56 - Connection status: True TXclient_alive: True 17:17:57 - Closing TIB connection to camera server 17:17:58 - Status: TibState.PpsStopped 17:17:58 - Alarms: AlarmVector.ReadError 17:17:58 - Connection status: False TXclient_alive: False 17:17:58 - Run duration: 0:00:20 17:17:58 - Local camera triggers: 0 17:17:58 - Slow control triggers: 0 17:17:58 - Single photo-electron triggers: 0 17:17:58 - Calibration triggers: 0 17:17:58 - Software triggers from UCTS: 0 17:17:58 - Pedestal triggers: 19902 17:17:58 - Collected input triggers: 18141 17:17:58 - Busy triggers: 0 17:17:58 - Camera event triggers sent to clusters: 18141 17:18:02 - Calling method FSM.gotoReady in object FSM 17:18:07 - Stopping NMC 17:18:12 - Found 0 PPS errors. 17:18:45 - NMC is in state NmcState.Configured and knows of 265 drawer(s) 17:18:59 - All front-end boards have counted 18141 event(s) 17:19:04 - Found 0 PPS errors.