17:01:28 - Starting task "Configure modules"
17:01:28 - Configuring modules with configuration /local/home/cta_test_bench/config/2020/20200227/config_tower_53modules.NMC.xml
17:01:28 - Switching off high voltages in configuration
17:01:31 - Successfully configured 53 module(s)
17:01:32 - All front-end boards are programmed with firmware F55D2001
17:01:33 - All backplanes are programmed with firmware 15
17:01:35 - Stopped periodic check of locked high voltages
17:01:38 - NMC is in state NmcState.Configured and knows of 53 drawer(s)
17:01:39 - Found 0 PPS errors.
17:01:39 - Finished task "Configure modules"
17:01:55 - High voltage locked channels will be checked every 30 s
17:02:24 - Checking for high voltage locked pixels...
17:02:30 - Starting task "Check high voltages"
17:02:31 - Found 0 drawer(s) in Nominal state, 0 in Low state and 53 in Wait state
17:02:39 - Actual voltage is between 210.45 V and 235.46 V.
17:02:39 - Low voltages range between 300.00 V and 300.00 V. 
17:02:48 - Reference voltages range between 945.26 V and 1167.15 V. 
17:02:53 - Maximum difference between expected and actual voltages is 214.11 V
17:02:54 - Checking for high voltage locked pixels...
17:02:58 - Maximum of current limits is 5647.07 uA
17:02:58 - Checking status of HVPA module in 53 drawer(s)...
17:02:58 - Calling method FSM.gotoObserving in object FSM
17:02:58 - Checking status of HVPA channels in 53 drawer(s)...
17:03:03 - HV is locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 371 channel(s)
17:03:03 - Finished task "Check high voltages"
17:03:24 - Checking for high voltage locked pixels...
17:03:33 - Starting task "Check high voltages"
17:03:33 - Found 0 drawer(s) in Nominal state, 0 in Low state and 53 in Wait state
17:03:41 - Actual voltage is between 163.48 V and 175.68 V.
17:03:42 - Low voltages range between 300.00 V and 300.00 V. 
17:03:51 - Reference voltages range between 945.26 V and 1167.15 V. 
17:03:54 - Checking for high voltage locked pixels...
17:03:56 - Maximum difference between expected and actual voltages is 167.14 V
17:03:56 - Stopped periodic check of locked high voltages
17:04:01 - Maximum of current limits is 5647.07 uA
17:04:01 - Checking status of HVPA module in 53 drawer(s)...
17:04:01 - Checking status of HVPA channels in 53 drawer(s)...
17:04:05 - HV is locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 371 channel(s)
17:04:05 - Finished task "Check high voltages"
17:04:06 - Starting task "Switch HVPA voltage state"
17:04:06 - Setting HVPA current limit to 12489.75 uA
17:04:24 - Setting HVPA ramp speed to 25 V/s
17:04:26 - Switching HVPA to state Nominal
17:04:35 - Setting HVPA current limits...
17:04:58 - Finished task "Switch HVPA voltage state"
17:05:18 - High voltage locked channels will be checked every 30 s
17:05:48 - Checking for high voltage locked pixels...
17:05:49 - Module 85 has 1 pixel(s) locked (HVPA status of 0x3f)!
17:05:49 - Module 94 has 1 pixel(s) locked (HVPA status of 0x7d)!
17:05:49 - Module 131 has 1 pixel(s) locked (HVPA status of 0x77)!
17:05:50 - Unlocking 3 channel(s)...
17:05:50 - Resetting HVPA channel 6 of drawer 85...
17:05:50 - Resetting HVPA channel 1 of drawer 94...
17:05:50 - Resetting HVPA channel 3 of drawer 131...
17:06:18 - Checking for high voltage locked pixels...
17:06:19 - Module 131 has 1 pixel(s) locked (HVPA status of 0x77)!
17:06:20 - Unlocking 1 channel(s)...
17:06:20 - Resetting HVPA channel 3 of drawer 131...
17:06:48 - Checking for high voltage locked pixels...
17:06:49 - Module 131 has 1 pixel(s) locked (HVPA status of 0x77)!
17:06:50 - Unlocking 1 channel(s)...
17:06:50 - Resetting HVPA channel 3 of drawer 131...
17:07:01 - Starting task "Check high voltages"
17:07:02 - Found 53 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
17:07:10 - Actual voltage is between 87.23 V and 1167.54 V.
17:07:11 - Low voltages range between 300.00 V and 300.00 V. 
17:07:18 - Checking for high voltage locked pixels...
17:07:19 - Module 131 has 1 pixel(s) locked (HVPA status of 0x77)!
17:07:20 - Unlocking 1 channel(s)...
17:07:20 - Resetting HVPA channel 3 of drawer 131...
17:07:20 - Reference voltages range between 0.00 V and 1167.15 V. 
17:07:29 - Maximum difference between expected and actual voltages is 82.35 V
17:07:33 - Maximum of current limits is 5647.07 uA
17:07:33 - Checking status of HVPA module in 53 drawer(s)...
17:07:34 - HVPA module in drawer 131 has a status of 0x77!
17:07:34 - Checking status of HVPA channels in 53 drawer(s)...
17:07:36 - HVPA channel 3 of drawer 131 has status 0x8003!
17:07:38 - HV is locked in 1 channel(s), was in rise default in 0 channel(s) and was still in progress in 0 channel(s)
17:07:38 - Resetting 1 channel(s)...
17:07:38 - Resetting HVPA channel 3 of drawer 131...
17:07:48 - Checking for high voltage locked pixels...
17:07:49 - Checking reference and low voltages against /local/home/cta_test_bench/config/2020/20200227/config_tower_53modules.NMC.xml
17:07:54 - Done
17:07:54 - Finished task "Check high voltages"
17:08:07 - Starting task "Check high voltages"
17:08:08 - Found 53 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
17:08:16 - Actual voltage is between 945.50 V and 1167.54 V.
17:08:17 - Low voltages range between 300.00 V and 300.00 V. 
17:08:18 - Checking for high voltage locked pixels...
17:08:26 - Reference voltages range between 945.26 V and 1167.15 V. 
17:08:35 - Maximum difference between expected and actual voltages is 0.71 V
17:08:39 - Maximum of current limits is 5647.07 uA
17:08:39 - Checking status of HVPA module in 53 drawer(s)...
17:08:40 - Checking status of HVPA channels in 53 drawer(s)...
17:08:44 - HV is stabilized in all channels.
17:08:44 - Checking reference and low voltages against /local/home/cta_test_bench/config/2020/20200227/config_tower_53modules.NMC.xml
17:08:48 - Checking for high voltage locked pixels...
17:08:49 - Done
17:08:49 - Finished task "Check high voltages"
17:09:18 - Checking for high voltage locked pixels...
17:09:38 - Stopping NMC
17:09:38 - Found 0 PPS errors.
17:09:39 - Starting NMC
17:09:39 - Clearing PPS error counters
17:09:43 - NMC is in state NmcState.Running and knows of 53 drawer(s)
17:09:44 - Found 0 PPS errors.
17:09:48 - Checking for high voltage locked pixels...
17:09:52 - Configuring UCTS
17:09:52 - Configuring TIB...
17:09:52 - Setting TIB forced mode to False
17:09:52 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
17:09:52 - Status: TibState.PpsStopped
17:09:52 - Alarms: AlarmVector.ReadError
17:09:52 - Connection status: False
17:09:52 - Entering set_external_delay
17:09:52 - Setting ExternalDelayLocal to 0.000 ns
17:09:52 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
17:09:52 - Entering set_external_delay
17:09:52 - Setting ExternalDelayCalibration to 65.000 ns
17:09:52 - Writing 65.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
17:09:52 - Entering set_external_delay
17:09:52 - Setting ExternalDelaySphe to 150.000 ns
17:09:52 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
17:09:53 - Setting TIB trigger mode to 0x54 = TriggerMask.SlowControl|Software|Calibration
17:09:53 - Writing 84 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
17:09:53 - Writing 0 to Camera.TIB.InputsEnable.InputMask.InputMask_v
17:09:53 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
17:09:53 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
17:09:53 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
17:09:53 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
17:09:53 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
17:09:53 - Status: TibState.PpsStopped
17:09:53 - Alarms: AlarmVector.ReadError
17:09:53 - Connection status: False
17:09:53 - Writing False to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
17:09:53 - Writing 100 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
17:09:53 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
17:09:53 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
17:09:53 - Writing 100 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
17:09:53 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
17:09:53 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
17:09:53 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
17:09:53 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
17:09:53 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
17:09:54 - Connecting TIB to camera server at 10.10.4.12:222
17:09:54 - Starting TIB...
17:09:54 - Status: TibState.PpsStopped
17:09:54 - Alarms: AlarmVector.ReadError
17:09:54 - Connection status: False
17:09:54 - Resetting run
17:09:54 - Wait for UCTS external trigger before enabling trigger...
17:09:54 - Status: TibState.Standby
17:09:54 - Alarms: AlarmVector.NoAlarm
17:09:54 - Connection status: False
17:09:54 - Status: TibState.Standby
17:09:54 - Alarms: AlarmVector.NoAlarm
17:09:54 - Connection status: False
17:09:55 - Status: TibState.Armed
17:09:55 - Alarms: AlarmVector.NoAlarm
17:09:55 - Connection status: True
17:09:57 - Starting TIB triggers...
17:09:57 - Enabling trigger
17:09:57 - Status: TibState.Running
17:09:57 - Alarms: AlarmVector.NoAlarm
17:09:57 - Connection status: True
17:10:00 - Configuring calibration box...
17:10:00 - Configuring calibration box with leds=0x358 voltage=16.00 frequency=1000.0/1 central_current=0.0 duration=120 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=False enabled=False
17:10:02 - Starting flashes...
17:10:18 - Stopping TIB...
17:10:18 - Status: TibState.Running
17:10:18 - Alarms: AlarmVector.ReadError
17:10:18 - Connection status: True
17:10:18 - Disabling trigger
17:10:18 - Disabling PPS
17:10:19 - Status: TibState.PpsStopped
17:10:19 - Alarms: AlarmVector.ReadError
17:10:19 - Connection status: True
17:10:19 - Closing TIB connection to camera server
17:10:19 - Checking for high voltage locked pixels...
17:10:23 - Calling method FSM.gotoReady in object FSM
17:10:29 - Status: TibState.PpsStopped
17:10:29 - Alarms: AlarmVector.ReadError
17:10:29 - Connection status: False
17:10:29 - Run duration: 0:00:22
17:10:29 - Local camera triggers: 0
17:10:29 - Slow control triggers: 0
17:10:29 - Single photo-electron triggers: 0
17:10:29 - Calibration triggers: 11955
17:10:29 - Software triggers from UCTS: 0
17:10:29 - Pedestal triggers: 0
17:10:29 - Collected input triggers: 11955
17:10:29 - Busy triggers: 0
17:10:29 - Camera event triggers sent to clusters: 11955