15:38:56 - Setting HVPA ramp speed to 500 V/s
15:38:58 - Switching HVPA to state Wait
15:39:07 - Setting HVPA current limits...
15:39:30 - Finished task "Switch HVPA voltage state"
15:40:34 - Starting task "Check high voltages"
15:40:35 - Found 0 drawer(s) in Nominal state, 0 in Low state and 53 in Wait state
15:40:43 - Actual voltage is between 164.09 V and 181.17 V.
15:40:44 - Low voltages range between 300.00 V and 300.00 V. 
15:40:53 - Reference voltages range between 945.26 V and 1167.15 V. 
15:40:58 - Maximum difference between expected and actual voltages is 172.02 V
15:41:02 - Maximum of current limits is 5647.07 uA
15:41:02 - Checking status of HVPA module in 53 drawer(s)...
15:41:03 - Checking status of HVPA channels in 53 drawer(s)...
15:41:07 - HV is locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 371 channel(s)
15:41:07 - Finished task "Check high voltages"
15:41:10 - Resetting NMC
15:41:14 - Starting task "Powering modules..."
15:41:14 - Executing command '/opt/cta/bin/cta_ctrl --all --power_off' on Digital Trigger Crate at 10.10.4.57
15:41:16 - Finished task "Powering modules..."
15:41:21 - ECC is in state EccState.Ready with status EccStatus.NoError
15:41:21 - Average temperature: 19.6 deg C
15:41:21 - Air temperature (T1-T6 and T9-T10): 19.6, -838.0, 1254.0, -838.0, 1254.0, 1254.0, 1254.0, 1253.0 deg C
15:41:21 - Air differential pressure: 0.0 Pa
15:41:21 - Water temperature (T11-T12): -838.0, 1254.0 deg C
15:41:21 - Back doors are closed.
15:41:21 - Shutter is in state PartiallyOpen (error 0)
15:41:21 - Light sensors: [2.28, 2.74, 3.17, 3.5] V
15:41:21 - Humidity: 0.0 %. Water pressure: [0.0, 0.0] bar.
15:41:21 - Fan speeds: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0] rpm
15:41:21 - 24 V power supply no. 1 is enabled with a current of 0.340 A.
15:42:02 - Starting task "Powering modules..."
15:42:02 - Executing command '/opt/cta/bin/cta_ctrl --slot 1 --all --power_on' on Digital Trigger Crate at 10.10.4.57
15:42:08 - Executing command '/opt/cta/bin/cta_ctrl --slot 5 --all --power_on' on Digital Trigger Crate at 10.10.4.57
15:42:13 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --all --power_on' on Digital Trigger Crate at 10.10.4.57
15:42:19 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --all --power_on' on Digital Trigger Crate at 10.10.4.57
15:42:25 - Executing command '/opt/cta/bin/cta_ctrl --slot 13 --all --power_on' on Digital Trigger Crate at 10.10.4.57
15:42:31 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --all --power_on' on Digital Trigger Crate at 10.10.4.57
15:42:36 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --all --power_on' on Digital Trigger Crate at 10.10.4.57
15:42:42 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --all --power_on' on Digital Trigger Crate at 10.10.4.57
15:42:43 - Scanning interface p4p1 for non configured modules...
15:42:43 - Found 48 MAC address(es)
15:42:46 - Scanning interface p4p1 for non configured modules...
15:42:46 - Found 51 MAC address(es)
15:42:48 - Finished task "Powering modules..."
15:42:50 - Scanning interface p4p1 for non configured modules...
15:42:50 - Found 61 MAC address(es)
15:42:55 - Starting task "Configure modules"
15:42:55 - Configuring modules with configuration /local/home/cta_test_bench/config/2020/20200610/config_tower_53modules.NMC.xml
15:42:55 - Switching off high voltages in configuration
15:42:58 - Successfully configured 53 module(s)
15:42:59 - All front-end boards are programmed with firmware F55D2001
15:43:00 - All backplanes are programmed with firmware 15
15:43:05 - NMC is in state NmcState.Configured and knows of 53 drawer(s)
15:43:06 - Found 0 PPS errors.
15:43:06 - Finished task "Configure modules"
15:43:09 - Starting task "Switch HVPA voltage state"
15:43:09 - Setting HVPA current limit to 12489.75 uA
15:43:13 - Starting task "Power off channels not corresponding to a configured module"
15:43:13 - Powering off all but 53 DTC channel(s)
15:43:14 - Executing command '/opt/cta/bin/cta_ctrl --slot 1 --channel 4 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:15 - Executing command '/opt/cta/bin/cta_ctrl --slot 1 --channel 11 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:17 - Executing command '/opt/cta/bin/cta_ctrl --slot 5 --channel 3 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:19 - Executing command '/opt/cta/bin/cta_ctrl --slot 5 --channel 6 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:21 - Executing command '/opt/cta/bin/cta_ctrl --slot 5 --channel 9 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:22 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 1 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:24 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 2 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:26 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 4 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:27 - Setting HVPA ramp speed to 25 V/s
15:43:28 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 6 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:29 - Switching HVPA to state Nominal
15:43:29 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 7 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:31 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 8 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:32 - High voltage locked channels will be checked every 30 s
15:43:33 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 9 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:35 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 10 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:36 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 11 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:38 - Setting HVPA current limits...
15:43:38 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 12 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:40 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 13 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:42 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 14 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:43 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --channel 15 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:45 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 1 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:47 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 2 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:49 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 3 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:50 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 4 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:52 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 5 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:54 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 6 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:56 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 7 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:57 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 8 --power_off' on Digital Trigger Crate at 10.10.4.57
15:43:59 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 9 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:01 - Finished task "Switch HVPA voltage state"
15:44:01 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 10 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:02 - Checking for high voltage locked pixels...
15:44:03 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 11 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:04 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 12 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:06 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 13 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:08 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 14 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:10 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --channel 15 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:11 - Executing command '/opt/cta/bin/cta_ctrl --slot 13 --channel 1 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:13 - Executing command '/opt/cta/bin/cta_ctrl --slot 13 --channel 2 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:15 - Executing command '/opt/cta/bin/cta_ctrl --slot 13 --channel 13 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:17 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 1 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:18 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 2 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:20 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 3 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:22 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 5 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:24 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 8 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:25 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 9 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:27 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 10 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:29 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 11 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:31 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 12 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:32 - Checking for high voltage locked pixels...
15:44:32 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 13 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:34 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 14 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:36 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --channel 15 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:38 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 1 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:38 - Starting task "Check high voltages"
15:44:39 - Found 53 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
15:44:39 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 2 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:40 - Starting NMC
15:44:40 - Clearing PPS error counters
15:44:41 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 3 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:43 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 4 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:45 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 5 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:46 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 6 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:47 - Actual voltage is between 945.50 V and 1167.54 V.
15:44:48 - Low voltages range between 300.00 V and 300.00 V. 
15:44:48 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 7 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:50 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 8 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:52 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 11 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:53 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 12 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:55 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 13 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:57 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 14 --power_off' on Digital Trigger Crate at 10.10.4.57
15:44:57 - Reference voltages range between 945.26 V and 1167.15 V. 
15:44:58 - Calling method FSM.gotoObserving in object FSM
15:44:59 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --channel 15 --power_off' on Digital Trigger Crate at 10.10.4.57
15:45:00 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 3 --power_off' on Digital Trigger Crate at 10.10.4.57
15:45:02 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 4 --power_off' on Digital Trigger Crate at 10.10.4.57
15:45:02 - Checking for high voltage locked pixels...
15:45:04 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 5 --power_off' on Digital Trigger Crate at 10.10.4.57
15:45:06 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 6 --power_off' on Digital Trigger Crate at 10.10.4.57
15:45:06 - Maximum difference between expected and actual voltages is 0.72 V
15:45:07 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 11 --power_off' on Digital Trigger Crate at 10.10.4.57
15:45:09 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 14 --power_off' on Digital Trigger Crate at 10.10.4.57
15:45:11 - Maximum of current limits is 5647.07 uA
15:45:11 - Checking status of HVPA module in 53 drawer(s)...
15:45:11 - Finished task "Power off channels not corresponding to a configured module"
15:45:11 - Checking status of HVPA channels in 53 drawer(s)...
15:45:12 - Configuring calibration box...
15:45:12 - Configuring calibration box with leds=0x40 voltage=10.00 frequency=1000.0/1 central_current=0.0 duration=300 width=1 lemo_out=True fiber1_out=True fiber2_out=False lemo_in=False enabled=False
15:45:15 - HV is stabilized in all channels.
15:45:15 - Checking reference and low voltages against /local/home/cta_test_bench/config/2020/20200610/config_tower_53modules.NMC.xml
15:45:20 - Done
15:45:21 - Finished task "Check high voltages"
15:45:22 - Configuring UCTS
15:45:22 - Configuring TIB...
15:45:22 - Setting TIB forced mode to False
15:45:22 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
15:45:22 - Status: TibState.PpsStopped
15:45:22 - Alarms: AlarmVector.ReadError
15:45:22 - Connection status: False TXclient_alive: False
15:45:22 - Entering set_external_delay
15:45:22 - Setting ExternalDelayLocal to 0.000 ns
15:45:22 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
15:45:22 - Entering set_external_delay
15:45:22 - Setting ExternalDelayCalibration to 65.000 ns
15:45:22 - Writing 65.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
15:45:23 - Entering set_external_delay
15:45:23 - Setting ExternalDelaySphe to 150.000 ns
15:45:23 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
15:45:23 - Setting TIB trigger mode to 0x74 = TriggerMask.SlowControl|Pedestal|Software|Calibration
15:45:23 - Writing 116 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
15:45:23 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v
15:45:23 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
15:45:23 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
15:45:23 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
15:45:23 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
15:45:23 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
15:45:23 - Status: TibState.PpsStopped
15:45:23 - Alarms: AlarmVector.ReadError
15:45:23 - Connection status: False TXclient_alive: False
15:45:23 - Writing True to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
15:45:23 - Writing 100 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
15:45:23 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:45:23 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
15:45:23 - Writing 100 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
15:45:23 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
15:45:23 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:45:23 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
15:45:23 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
15:45:23 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:45:24 - Connecting TIB to camera server at 10.10.4.12:222
15:45:24 - Starting TIB...
15:45:24 - Status: TibState.PpsStopped
15:45:24 - Alarms: AlarmVector.ReadError
15:45:24 - Connection status: False TXclient_alive: False
15:45:24 - Resetting run
15:45:24 - Wait for UCTS external trigger before enabling trigger...
15:45:24 - Status: TibState.Standby
15:45:24 - Alarms: AlarmVector.NoAlarm
15:45:24 - Connection status: False TXclient_alive: True
15:45:24 - Status: TibState.Standby
15:45:24 - Alarms: AlarmVector.NoAlarm
15:45:24 - Connection status: False TXclient_alive: True
15:45:25 - Status: TibState.Armed
15:45:25 - Alarms: AlarmVector.NoAlarm
15:45:25 - Connection status: True TXclient_alive: True
15:45:27 - Starting TIB triggers...
15:45:27 - Enabling trigger
15:45:27 - Status: TibState.Running
15:45:27 - Alarms: AlarmVector.NoAlarm
15:45:27 - Connection status: True TXclient_alive: True
15:45:29 - Starting flashes...
15:45:32 - Checking for high voltage locked pixels...
15:46:02 - Checking for high voltage locked pixels...
15:46:04 - Stopping TIB...
15:46:04 - Status: TibState.Running
15:46:04 - Alarms: AlarmVector.ReadError
15:46:04 - Connection status: True TXclient_alive: True
15:46:04 - Disabling trigger
15:46:04 - Disabling PPS
15:46:05 - Status: TibState.PpsStopped
15:46:05 - Alarms: AlarmVector.ReadError
15:46:05 - Connection status: True TXclient_alive: True
15:46:05 - Closing TIB connection to camera server
15:46:05 - Status: TibState.PpsStopped
15:46:05 - Alarms: AlarmVector.ReadError
15:46:05 - Connection status: False TXclient_alive: False
15:46:05 - Run duration: 0:00:39
15:46:05 - Local camera triggers: 1
15:46:05 - Slow control triggers: 0
15:46:05 - Single photo-electron triggers: 0
15:46:05 - Calibration triggers: 29978
15:46:05 - Software triggers from UCTS: 0
15:46:05 - Pedestal triggers: 3869
15:46:05 - Collected input triggers: 33665
15:46:05 - Busy triggers: 47
15:46:05 - Camera event triggers sent to clusters: 33618
15:46:08 - Calling method FSM.gotoReady in object FSM