16:41:55 - ECC is in state EccState.Ready and transition EccTransition.Idle
16:41:55 - Active alarms: ['Alarm_General_current']
16:41:55 - PDB temperature: 29.7 deg C
16:41:55 - Status of UPS power: V=233.0 V, I=0.90 A, P=0.090 kW, Q=0.000 kVAR, S=0.210 kVA, cos=-0.40
16:41:55 - Status of 400 V: V1=233.0 V, V2=232.0 V, V3=232.0 V, I1=2.45 A, I2=5.11 A, I3=1.33 A, f=50.0 Hz, cos=0.74
16:41:55 - Status of 400 V: P1=0.020 kW, P2=0.800 kW, P3=-0.100 kW, Q1=0.450 kVAR, Q2=-0.100 kVAR, Q3=0.250 kVAR, S1=0.560 kVA, S2=1.170 kVA, S3=-0.300 kVA
16:41:55 - 2 relays enabled in PSB1. Total current of 0.0 A.
16:41:55 - 1 relays enabled in PSB2. Total current of 0.5 A.
16:41:55 - Status of PDB switches: StateBreaker=True, StateEmergencyStop=False, SwitchStatus_General400V=True, SwitchStatus_General230V=False, SwitchStatus_line400V=True, SwitchStatus_switch1=True, SwitchStatus_switch2=True, SwitchStatus_fanFront=True, SwitchStatus_fanRear=True, SwitchStatus_shutter=False
16:41:55 - Average air temperature: 24.0 deg C
16:41:55 - Air temperature (T1-T13): 24.0, 24.7, 25.8, 25.0, 22.8, 23.8, 22.7, 23.8, 24.6, 26.1, 23.4, 21.2, 21.2 deg C
16:41:55 - Air differential pressure: 0.2 Pa
16:41:55 - Back doors are closed.
16:41:55 - Light sensors: [0.0, 0.0] V
16:41:55 - Cooling valve is 100 %% open.Water flow: 29.5 l/min.Water pressure: [1.0, 1.5] bar
16:41:55 - Water temperature (T14-T16): 18.0, 15.8, 15.5 deg C
16:41:55 - Humidity: 45.2 %. 
16:41:55 - Fan speeds: [2460.0, 2400.0, 2400.0, 2400.0, 2460.0, 2460.0, 1680.0, 1560.0, 1560.0, 1560.0, 1500.0, 1560.0, 1560.0, 1260.0, 7560.0, 7500.0] rpm
16:41:59 - Starting task "Powering modules..."
16:41:59 - Enabling power in all modules
16:41:59 - Finished task "Powering modules..."
16:42:02 - Updating DTC module currents...
16:42:12 - Starting task "Configure modules"
16:42:12 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.6.0-41551 
16:42:12 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.6.0-41551 
16:42:12 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.6.0-41551 
16:42:12 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.6.0-41551
16:42:12 - Configuring modules with configuration /data/config/2021/20211011/config_cameraqm_265febs_newHV3.NMC.xml
16:42:12 - Switching off high voltages in configuration
16:42:22 - Successfully configured 265 module(s)
16:42:22 - Waiting for modules to be configured...
16:42:29 - All front-end boards are programmed with firmware F56D2002
16:42:42 - Found 2 different versions of DTBP firmware: [15 17]
16:43:41 - NMC is in state NmcState.Ready and knows of 265 drawer(s)
16:44:04 - All front-end boards have counted 0 event(s)
16:44:12 - Found PPS errors: 212 in drawer 0, 214 in drawer 1, 212 in drawer 2, 212 in drawer 3, 212 in drawer 4, 212 in drawer 5, 212 in drawer 8, 212 in drawer 9, 212 in drawer 10, 212 in drawer 11, 214 in drawer 12, 212 in drawer 13, 212 in drawer 14, 214 in drawer 15, 214 in drawer 16, 214 in drawer 17, 214 in drawer 18, 213 in drawer 19, 213 in drawer 20, 213 in drawer 21, 215 in drawer 22, 213 in drawer 23, 213 in drawer 24, 213 in drawer 25, 213 in drawer 26, 216 in drawer 27, 214 in drawer 28, 216 in drawer 29, 216 in drawer 30, 214 in drawer 31, 216 in drawer 32, 214 in drawer 33, 214 in drawer 34, 216 in drawer 35, 214 in drawer 36, 216 in drawer 37, 214 in drawer 38, 214 in drawer 39, 216 in drawer 40, 214 in drawer 41, 216 in drawer 42, 214 in drawer 43, 216 in drawer 44, 214 in drawer 45, 214 in drawer 46, 216 in drawer 47, 216 in drawer 48, 214 in drawer 49, 214 in drawer 50, 214 in drawer 51, 214 in drawer 52, 215 in drawer 53, 215 in drawer 54, 215 in drawer 55, 215 in drawer 56, 215 in drawer 57, 217 in drawer 58, 215 in drawer 59, 215 in drawer 60, 215 in drawer 61, 216 in drawer 62, 216 in drawer 63, 216 in drawer 64, 216 in drawer 65, 218 in drawer 66, 216 in drawer 67, 216 in drawer 68, 216 in drawer 69, 216 in drawer 70, 216 in drawer 71, 216 in drawer 72, 216 in drawer 73, 216 in drawer 74, 218 in drawer 75, 216 in drawer 76, 216 in drawer 77, 218 in drawer 78, 216 in drawer 79, 216 in drawer 80, 216 in drawer 81, 218 in drawer 82, 218 in drawer 83, 216 in drawer 84, 216 in drawer 85, 216 in drawer 86, 218 in drawer 87, 217 in drawer 88, 217 in drawer 89, 217 in drawer 90, 219 in drawer 91, 219 in drawer 92, 217 in drawer 93, 217 in drawer 94, 219 in drawer 95, 217 in drawer 96, 218 in drawer 97, 220 in drawer 98, 220 in drawer 99, 218 in drawer 100, 218 in drawer 101, 218 in drawer 102, 220 in drawer 103, 220 in drawer 104, 218 in drawer 105, 218 in drawer 106, 218 in drawer 107, 218 in drawer 108, 218 in drawer 109, 218 in drawer 110, 218 in drawer 111, 220 in drawer 112, 220 in drawer 113, 220 in drawer 114, 218 in drawer 115, 220 in drawer 116, 218 in drawer 117, 218 in drawer 118, 218 in drawer 119, 218 in drawer 120, 218 in drawer 121, 218 in drawer 122, 219 in drawer 123, 221 in drawer 124, 221 in drawer 125, 219 in drawer 126, 219 in drawer 127, 219 in drawer 128, 219 in drawer 129, 219 in drawer 130, 219 in drawer 131, 220 in drawer 132, 222 in drawer 133, 220 in drawer 134, 220 in drawer 135, 222 in drawer 136, 222 in drawer 137, 220 in drawer 138, 220 in drawer 139, 220 in drawer 140, 220 in drawer 141, 220 in drawer 142, 222 in drawer 143, 220 in drawer 144, 220 in drawer 145, 220 in drawer 146, 220 in drawer 147, 220 in drawer 148, 222 in drawer 149, 220 in drawer 150, 220 in drawer 151, 220 in drawer 152, 220 in drawer 153, 220 in drawer 154, 220 in drawer 155, 220 in drawer 156, 222 in drawer 157, 221 in drawer 158, 221 in drawer 159, 223 in drawer 160, 223 in drawer 161, 221 in drawer 162, 221 in drawer 163, 221 in drawer 164, 223 in drawer 165, 223 in drawer 166, 222 in drawer 167, 222 in drawer 168, 222 in drawer 169, 222 in drawer 170, 222 in drawer 171, 224 in drawer 172, 222 in drawer 173, 222 in drawer 174, 224 in drawer 175, 222 in drawer 176, 224 in drawer 177, 222 in drawer 178, 222 in drawer 179, 222 in drawer 180, 222 in drawer 181, 222 in drawer 182, 222 in drawer 183, 222 in drawer 184, 224 in drawer 185, 222 in drawer 186, 222 in drawer 187, 222 in drawer 188, 222 in drawer 189, 222 in drawer 190, 222 in drawer 191, 222 in drawer 192, 223 in drawer 193, 223 in drawer 194, 223 in drawer 195, 223 in drawer 196, 223 in drawer 197, 223 in drawer 198, 225 in drawer 199, 223 in drawer 200, 223 in drawer 201, 224 in drawer 202, 224 in drawer 203, 224 in drawer 204, 224 in drawer 205, 224 in drawer 206, 226 in drawer 207, 224 in drawer 208, 224 in drawer 209, 224 in drawer 210, 224 in drawer 211, 224 in drawer 212, 226 in drawer 213, 224 in drawer 214, 224 in drawer 215, 224 in drawer 216, 224 in drawer 217, 224 in drawer 218, 224 in drawer 219, 224 in drawer 220, 226 in drawer 221, 224 in drawer 222, 224 in drawer 223, 224 in drawer 224, 224 in drawer 225, 224 in drawer 226, 224 in drawer 227, 225 in drawer 228, 225 in drawer 229, 225 in drawer 230, 225 in drawer 231, 225 in drawer 232, 225 in drawer 233, 225 in drawer 234, 225 in drawer 235, 226 in drawer 236, 226 in drawer 237, 226 in drawer 238, 226 in drawer 239, 226 in drawer 240, 226 in drawer 241, 226 in drawer 242, 226 in drawer 243, 226 in drawer 244, 226 in drawer 245, 226 in drawer 246, 226 in drawer 247, 226 in drawer 248, 226 in drawer 249, 226 in drawer 250, 226 in drawer 251, 226 in drawer 252, 226 in drawer 253, 226 in drawer 254, 226 in drawer 255, 226 in drawer 256, 226 in drawer 257, 226 in drawer 258, 226 in drawer 259, 226 in drawer 260, 226 in drawer 261, 226 in drawer 262, 227 in drawer 263, 227 in drawer 264
16:44:12 - Finished task "Configure modules"
16:51:10 - Starting task "Switch HVPA voltage state"
16:51:10 - Setting HVPA current limit to 12489.75 uA
16:51:10 - Setting Hvpa.Hvpa0.CurrentTrip in all drawers
16:51:26 - Setting Hvpa.Hvpa1.CurrentTrip in all drawers
16:51:41 - Setting Hvpa.Hvpa2.CurrentTrip in all drawers
16:51:56 - Setting Hvpa.Hvpa3.CurrentTrip in all drawers
16:52:11 - Setting Hvpa.Hvpa4.CurrentTrip in all drawers
16:52:26 - Setting Hvpa.Hvpa5.CurrentTrip in all drawers
16:52:41 - Setting Hvpa.Hvpa6.CurrentTrip in all drawers
16:52:57 - Setting HVPA ramp speed to 25 V/s
16:53:14 - Switching HVPA to state Nominal
16:53:44 - High voltage locked channels will be checked every 60 s
16:53:46 - Setting HVPA current limits...
16:54:44 - Checking for high voltage locked pixels...
16:55:09 - HVPA channel 2 of drawer 86 has status 0x0007!
16:55:09 - HVPA channel 2 in module 86 is in rise default!
16:55:21 - HVPA channel 4 in module 153 is still in progress
16:55:32 - HVPA channel 6 in module 73 is still in progress
16:55:32 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 3 channel(s)
16:55:44 - Checking for high voltage locked pixels...
16:56:09 - HVPA channel 2 of drawer 86 has status 0x0007!
16:56:09 - HVPA channel 2 in module 86 is in rise default!
16:56:32 - HVPA channel 6 in module 73 is still in progress
16:56:32 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 2 channel(s)
16:55:46 - Finished task "Switch HVPA voltage state"
16:56:32 - Starting NMC
16:56:32 - Clearing PPS error counters
16:56:54 - Resetting FEB counters...
16:57:00 - Starting all configured modules...
16:57:08 - Checking for high voltage locked pixels...
16:57:33 - HVPA channel 2 of drawer 86 has status 0x0007!
16:57:33 - HVPA channel 2 in module 86 is in rise default!
16:57:56 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 1 channel(s)
16:57:56 - Checking for high voltage locked pixels...
16:58:21 - HVPA channel 2 of drawer 86 has status 0x0007!
16:58:22 - HVPA channel 2 in module 86 is in rise default!
16:58:45 - HVPA channel 6 in module 73 is still in progress
16:58:45 - HVPA channel 6 in module 211 is still in progress
16:58:45 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 3 channel(s)
16:58:45 - Checking for high voltage locked pixels...
16:59:10 - HVPA channel 2 of drawer 86 has status 0x0007!
16:59:10 - HVPA channel 2 in module 86 is in rise default!
16:59:21 - HVPA channel 4 in module 106 is still in progress
16:59:27 - HVPA channel 5 in module 73 is still in progress
16:59:33 - HVPA channel 6 in module 73 is still in progress
16:59:33 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 4 channel(s)
16:59:36 - Stopped periodic check of locked high voltages
16:59:39 - Calling method FSM.gotoObserving in object FSM
16:59:39 - Starting run 2675
16:59:49 - Checking UCTS clock...
16:59:50 - UCTS clock is late by 9 seconds!
16:59:50 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9)
16:59:50 - Configuring TIB...
16:59:50 - Setting TIB forced mode to False
16:59:50 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
16:59:50 - Status: TibState.RunStopped
16:59:50 - Alarms: AlarmVector.BufferFull
16:59:50 - Connection status: False TXclient_alive: False
16:59:50 - Disabling PPS
16:59:52 - Status: TibState.PpsStopped
16:59:52 - Alarms: AlarmVector.BufferFull
16:59:52 - Connection status: False TXclient_alive: False
16:59:52 - Entering set_external_delay
16:59:52 - Setting ExternalDelayLocal to 27.000 ns
16:59:52 - Writing 27.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
16:59:53 - Entering set_external_delay
16:59:53 - Setting ExternalDelayCalibration to 58.000 ns
16:59:53 - Writing 58.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
16:59:53 - Entering set_external_delay
16:59:53 - Setting ExternalDelaySphe to 110.000 ns
16:59:53 - Writing 110.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
16:59:53 - Setting TIB trigger mode to 0x54 = TriggerMask.SlowControl|Software|Calibration
16:59:53 - Writing 84 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
16:59:53 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v
16:59:53 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
16:59:53 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
16:59:53 - Writing True to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
16:59:53 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
16:59:53 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
16:59:53 - Status: TibState.PpsStopped
16:59:53 - Alarms: AlarmVector.BufferFull
16:59:53 - Connection status: False TXclient_alive: False
16:59:53 - Writing False to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
16:59:53 - Writing 100 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
16:59:53 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:59:53 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
16:59:53 - Writing 100 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
16:59:53 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
16:59:53 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:59:53 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
16:59:53 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
16:59:53 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:59:55 - Connecting TIB to camera server at 10.10.4.12:222
16:59:55 - Starting TIB...
16:59:55 - Status: TibState.PpsStopped
16:59:55 - Alarms: AlarmVector.BufferFull
16:59:55 - Connection status: False TXclient_alive: False
16:59:55 - Resetting run
16:59:55 - Wait for UCTS external trigger before enabling trigger...
16:59:55 - Status: TibState.Standby
16:59:55 - Alarms: AlarmVector.NoAlarm
16:59:55 - Connection status: False TXclient_alive: True
16:59:55 - Status: TibState.Standby
16:59:55 - Alarms: AlarmVector.NoAlarm
16:59:55 - Connection status: False TXclient_alive: True
16:59:56 - Status: TibState.Armed
16:59:56 - Alarms: AlarmVector.NoAlarm
16:59:56 - Connection status: False TXclient_alive: True
16:59:57 - Starting TIB triggers...
16:59:57 - Enabling trigger
16:59:57 - Status: TibState.Running
16:59:57 - Alarms: AlarmVector.NoAlarm
16:59:57 - Connection status: True TXclient_alive: True
17:00:01 - Checking status of calibration box...
17:00:02 - Calibration box is configured with 1 leds at a frequency of 1000.0 Hz and a voltage of 10.99 V
17:00:07 - Configuring calibration box...
17:00:07 - Configuring calibration box with leds=0x40 voltage=11.00 frequency=1000.0/1 central_current=0.0 duration=500 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=False enabled=False
17:00:09 - Starting flashes...
17:01:08 - Stopping TIB...
17:01:08 - Status: TibState.Running
17:01:08 - Alarms: AlarmVector.ReadError
17:01:08 - Connection status: True TXclient_alive: True
17:01:08 - Disabling trigger
17:01:08 - Disabling PPS
17:01:09 - Status: TibState.PpsStopped
17:01:09 - Alarms: AlarmVector.ReadError
17:01:09 - Connection status: True TXclient_alive: True
17:01:10 - Closing TIB connection to camera server
17:01:11 - Status: TibState.PpsStopped
17:01:11 - Alarms: AlarmVector.ReadError
17:01:12 - Connection status: False TXclient_alive: False
17:01:12 - Run duration: 0:01:12
17:01:12 - Local camera triggers: 0
17:01:12 - Slow control triggers: 0
17:01:12 - Single photo-electron triggers: 0
17:01:12 - Calibration triggers: 49965
17:01:12 - Software triggers from UCTS: 0
17:01:12 - Pedestal triggers: 0
17:01:12 - Collected input triggers: 49965
17:01:12 - Busy triggers: 0
17:01:12 - Camera event triggers sent to clusters: 49965
17:01:14 - Calling method FSM.gotoReady in object FSM
17:01:17 - Stopping NMC
17:01:25 - Found 0 PPS errors.
17:02:13 - Calling method FSM.gotoReady in object FSM
17:02:13 - Failed to call FSM.gotoReady: Bad statuses are present