09:43:48 - Starting task "Configure modules"
09:43:49 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.5.1-40061 
09:43:49 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.5.1-40061 
09:43:49 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.5.1-40061 
09:43:49 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.5.1-40061
09:43:49 - Configuring modules with configuration /data/config/2020/20201208/config_tower_2modules_FEBv6_pingpong_newDacl_DelayBusy15.NMC.xml
09:43:49 - Switching off high voltages in configuration
09:43:52 - Successfully configured 2 module(s)
09:43:53 - All front-end boards are programmed with firmware 61201204
09:43:53 - All backplanes are programmed with firmware 15
09:43:53 - NMC is in state NmcState.Configured and knows of 2 drawer(s)
09:43:53 - Found 0 PPS errors.
09:43:54 - Finished task "Configure modules"
09:43:56 - Starting task "Switch HVPA voltage state"
09:43:56 - Setting HVPA current limit to 12489.75 uA
09:43:57 - Setting HVPA ramp speed to 25 V/s
09:43:57 - Switching HVPA to state Nominal
09:44:04 - Setting HVPA current limits...
09:44:05 - Finished task "Switch HVPA voltage state"
09:44:06 - High voltage locked channels will be checked every 30 s
09:44:09 - Starting task "Check high voltages"
09:44:09 - Found 2 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
09:44:09 - Actual voltage is between 178.12 V and 1062.62 V.
09:44:09 - Low voltages range between 300.00 V and 300.00 V. 
09:44:10 - Reference voltages range between 0.00 V and 1077.36 V. 
09:44:10 - Maximum difference between expected and actual voltages is 704.65 V
09:44:10 - Maximum of current limits is 4840.96 uA
09:44:10 - Checking status of HVPA module in 2 drawer(s)...
09:44:10 - HVPA module in drawer 84 has a status of 0x3f!
09:44:10 - HVPA module in drawer 85 has a status of 0x3f!
09:44:10 - Checking status of HVPA channels in 2 drawer(s)...
09:44:11 - HVPA channel 6 of drawer 84 has status 0x8003!
09:44:11 - HVPA channel 6 of drawer 85 has status 0x8003!
09:44:11 - HV is locked in 2 channel(s), was in rise default in 0 channel(s) and was still in progress in 10 channel(s)
09:44:11 - Resetting 2 channel(s)...
09:44:11 - Resetting HVPA channel 6 of drawer 84...
09:44:12 - Resetting HVPA channel 6 of drawer 85...
09:44:23 - Checking reference and low voltages against /data/config/2020/20201208/config_tower_2modules_FEBv6_pingpong_newDacl_DelayBusy15.NMC.xml
09:44:23 - Reference voltage of channel 6 in drawer #85: expected 1168.35, read 0.00
09:44:23 - Done
09:44:23 - Finished task "Check high voltages"
09:44:32 - Starting task "Check high voltages"
09:44:32 - Found 2 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
09:44:33 - Actual voltage is between 167.75 V and 1077.26 V.
09:44:33 - Low voltages range between 300.00 V and 300.00 V. 
09:44:33 - Reference voltages range between 0.00 V and 1077.36 V. 
09:44:34 - Maximum difference between expected and actual voltages is 163.48 V
09:44:34 - Maximum of current limits is 4840.96 uA
09:44:34 - Checking status of HVPA module in 2 drawer(s)...
09:44:34 - HVPA module in drawer 85 has a status of 0x3f!
09:44:34 - Checking status of HVPA channels in 2 drawer(s)...
09:44:34 - HVPA channel 6 of drawer 85 has status 0x8003!
09:44:34 - HV is locked in 1 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
09:44:34 - Resetting 1 channel(s)...
09:44:35 - Resetting HVPA channel 6 of drawer 85...
09:44:37 - Checking for high voltage locked pixels...
09:44:39 - Updating voltage widget...
09:44:45 - Checking reference and low voltages against /data/config/2020/20201208/config_tower_2modules_FEBv6_pingpong_newDacl_DelayBusy15.NMC.xml
09:44:46 - Reference voltage of channel 6 in drawer #85: expected 1168.35, read 0.00
09:44:46 - Done
09:44:46 - Finished task "Check high voltages"
09:44:52 - Starting task "Check high voltages"
09:44:52 - Found 2 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
09:44:52 - Actual voltage is between 172.02 V and 1077.26 V.
09:44:52 - Low voltages range between 300.00 V and 300.00 V. 
09:44:53 - Reference voltages range between 0.00 V and 1077.36 V. 
09:44:53 - Maximum difference between expected and actual voltages is 167.14 V
09:44:53 - Maximum of current limits is 4840.96 uA
09:44:53 - Checking status of HVPA module in 2 drawer(s)...
09:44:54 - HVPA module in drawer 85 has a status of 0x3f!
09:44:54 - Checking status of HVPA channels in 2 drawer(s)...
09:44:54 - HVPA channel 6 of drawer 85 has status 0x8003!
09:44:54 - HV is locked in 1 channel(s), was in rise default in 0 channel(s) and was still in progress in 0 channel(s)
09:44:54 - Resetting 1 channel(s)...
09:44:54 - Resetting HVPA channel 6 of drawer 85...
09:44:59 - Updating voltage widget...
09:45:05 - Checking reference and low voltages against /data/config/2020/20201208/config_tower_2modules_FEBv6_pingpong_newDacl_DelayBusy15.NMC.xml
09:45:05 - Done
09:45:05 - Finished task "Check high voltages"
09:45:07 - Checking for high voltage locked pixels...
09:45:09 - Starting task "Check high voltages"
09:45:09 - Found 2 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
09:45:09 - Actual voltage is between 568.52 V and 1077.26 V.
09:45:09 - Low voltages range between 300.00 V and 300.00 V. 
09:45:10 - Reference voltages range between 989.87 V and 1168.35 V. 
09:45:10 - Maximum difference between expected and actual voltages is 552.25 V
09:45:11 - Maximum of current limits is 4840.96 uA
09:45:11 - Checking status of HVPA module in 2 drawer(s)...
09:45:11 - Checking status of HVPA channels in 2 drawer(s)...
09:45:11 - HV is locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
09:45:11 - Checking reference and low voltages against /data/config/2020/20201208/config_tower_2modules_FEBv6_pingpong_newDacl_DelayBusy15.NMC.xml
09:45:11 - Done
09:45:11 - Finished task "Check high voltages"
09:45:19 - Updating voltage widget...
09:45:23 - Starting task "Check high voltages"
09:45:23 - Found 2 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
09:45:23 - Actual voltage is between 989.42 V and 1124.84 V.
09:45:24 - Low voltages range between 300.00 V and 300.00 V. 
09:45:24 - Reference voltages range between 989.87 V and 1168.35 V. 
09:45:24 - Maximum difference between expected and actual voltages is 14.23 V
09:45:25 - Maximum of current limits is 4840.96 uA
09:45:25 - Checking status of HVPA module in 2 drawer(s)...
09:45:25 - Checking status of HVPA channels in 2 drawer(s)...
09:45:25 - HV is locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
09:45:25 - Checking reference and low voltages against /data/config/2020/20201208/config_tower_2modules_FEBv6_pingpong_newDacl_DelayBusy15.NMC.xml
09:45:25 - Done
09:45:26 - Finished task "Check high voltages"
09:45:30 - Starting task "Check high voltages"
09:45:30 - Found 2 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
09:45:31 - Actual voltage is between 989.42 V and 1168.15 V.
09:45:31 - Low voltages range between 300.00 V and 300.00 V. 
09:45:31 - Reference voltages range between 989.87 V and 1168.35 V. 
09:45:32 - Maximum difference between expected and actual voltages is 0.52 V
09:45:32 - Maximum of current limits is 4840.96 uA
09:45:32 - Checking status of HVPA module in 2 drawer(s)...
09:45:32 - Checking status of HVPA channels in 2 drawer(s)...
09:45:32 - HV is stabilized in all channels.
09:45:32 - Checking reference and low voltages against /data/config/2020/20201208/config_tower_2modules_FEBv6_pingpong_newDacl_DelayBusy15.NMC.xml
09:45:33 - Done
09:45:33 - Finished task "Check high voltages"
09:45:35 - Starting NMC
09:45:35 - Clearing PPS error counters
09:45:37 - Checking for high voltage locked pixels...
09:45:52 - Configuring calibration box...
09:45:52 - Configuring calibration box with leds=0x40 voltage=15.00 frequency=1000.0/1 central_current=0.0 duration=0 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=True enabled=True
09:45:56 - Checking UCTS clock...
09:45:56 - UCTS and control PC clocks differ by 3.3 seconds
09:45:56 - Configuring UCTS
09:45:56 - Configuring TIB...
09:45:56 - Setting TIB forced mode to False
09:45:56 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
09:45:56 - Status: TibState.RunStopped
09:45:56 - Alarms: AlarmVector.BufferFull
09:45:56 - Connection status: False TXclient_alive: False
09:45:56 - Disabling PPS
09:45:58 - Status: TibState.PpsStopped
09:45:58 - Alarms: AlarmVector.BufferFull
09:45:58 - Connection status: False TXclient_alive: False
09:45:58 - Entering set_external_delay
09:45:58 - Setting ExternalDelayLocal to 0.000 ns
09:45:58 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
09:45:59 - Entering set_external_delay
09:45:59 - Setting ExternalDelayCalibration to 65.000 ns
09:45:59 - Writing 65.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
09:45:59 - Entering set_external_delay
09:45:59 - Setting ExternalDelaySphe to 150.000 ns
09:45:59 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
09:45:59 - Setting TIB trigger mode to 0x74 = TriggerMask.SlowControl|Pedestal|Software|Calibration
09:45:59 - Writing 116 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
09:45:59 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v
09:45:59 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
09:45:59 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
09:45:59 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
09:45:59 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
09:46:00 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
09:46:00 - Status: TibState.PpsStopped
09:46:00 - Alarms: AlarmVector.BufferFull
09:46:00 - Connection status: False TXclient_alive: False
09:46:00 - Writing True to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
09:46:00 - Writing 100 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
09:46:00 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
09:46:00 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
09:46:00 - Writing 100 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
09:46:00 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
09:46:00 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
09:46:00 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
09:46:00 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
09:46:00 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
09:46:02 - Connecting TIB to camera server at 10.10.4.12:222
09:46:02 - Starting TIB...
09:46:02 - Status: TibState.PpsStopped
09:46:02 - Alarms: AlarmVector.BufferFull
09:46:02 - Connection status: False TXclient_alive: False
09:46:02 - Resetting run
09:46:02 - Wait for UCTS external trigger before enabling trigger...
09:46:02 - Status: TibState.Standby
09:46:02 - Alarms: AlarmVector.NoAlarm
09:46:02 - Connection status: False TXclient_alive: True
09:46:02 - Status: TibState.Standby
09:46:02 - Alarms: AlarmVector.NoAlarm
09:46:02 - Connection status: False TXclient_alive: True
09:46:03 - Status: TibState.Armed
09:46:03 - Alarms: AlarmVector.NoAlarm
09:46:03 - Connection status: True TXclient_alive: True
09:46:06 - Starting TIB triggers...
09:46:06 - Enabling trigger
09:46:06 - Status: TibState.Running
09:46:06 - Alarms: AlarmVector.NoAlarm
09:46:06 - Connection status: True TXclient_alive: True
09:46:07 - Checking for high voltage locked pixels...
09:46:09 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
09:46:09 - Alarms: AlarmVector.NoAlarm
09:46:09 - Temperature: 30.81
09:46:09 - Local camera rate: 0.0 Hz
09:46:09 - Stereo trigger rate: 0.0 Hz
09:46:09 - SPE trigger rate: 0.0 Hz
09:46:09 - Calibration trigger rate: 8983.0 Hz
09:46:09 - Software trigger rate: 0.0 Hz
09:46:09 - Pedestal trigger rate: 100.0 Hz
09:46:09 - Collected trigger rate: 9008.0 Hz
09:46:09 - Busy rate: 92.0 Hz
09:46:09 - Camera trigger rate: 8916.0 Hz
09:46:11 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
09:46:11 - Alarms: AlarmVector.NoAlarm
09:46:11 - Temperature: 30.94
09:46:11 - Local camera rate: 0.0 Hz
09:46:11 - Stereo trigger rate: 0.0 Hz
09:46:11 - SPE trigger rate: 0.0 Hz
09:46:11 - Calibration trigger rate: 8958.0 Hz
09:46:12 - Software trigger rate: 0.0 Hz
09:46:12 - Pedestal trigger rate: 100.0 Hz
09:46:12 - Collected trigger rate: 8991.0 Hz
09:46:12 - Busy rate: 91.0 Hz
09:46:12 - Camera trigger rate: 8900.0 Hz
09:46:37 - Checking for high voltage locked pixels...
09:47:05 - Stopping TIB...
09:47:05 - Status: TibState.Running
09:47:05 - Alarms: AlarmVector.NoAlarm
09:47:05 - Connection status: True TXclient_alive: True
09:47:05 - Disabling trigger
09:47:05 - Disabling PPS
09:47:06 - Status: TibState.PpsStopped
09:47:06 - Alarms: AlarmVector.NoAlarm
09:47:06 - Connection status: True TXclient_alive: True
09:47:06 - Closing TIB connection to camera server
09:47:07 - Checking for high voltage locked pixels...
09:47:08 - Status: TibState.PpsStopped
09:47:08 - Alarms: AlarmVector.NoAlarm
09:47:09 - Connection status: False TXclient_alive: False
09:47:09 - Run duration: 0:01:02
09:47:09 - Local camera triggers: 0
09:47:09 - Slow control triggers: 0
09:47:09 - Single photo-electron triggers: 0
09:47:09 - Calibration triggers: 551718
09:47:09 - Software triggers from UCTS: 0
09:47:09 - Pedestal triggers: 6162
09:47:09 - Collected input triggers: 532435
09:47:09 - Busy triggers: 5905
09:47:09 - Camera event triggers sent to clusters: 526530
09:47:21 - Calling method FSM.gotoReady in object FSM
09:47:24 - Stopping NMC
09:47:24 - Found 0 PPS errors.