16:43:07 - Starting task "Prepare ECC"
16:43:07 - Preparing ECC...
16:43:07 - Setting ECC temperature range for Ready state to [16.000000, 36.000000]
16:43:07 - Setting ECC mode to EccTransition.Ready
16:43:08 - Received ECC state change notification: EccState.Safe
16:43:13 - Received ECC state change notification: EccState.Ready
16:43:14 - Setting power supply box SPE relay to True
16:43:14 - Finished task "Prepare ECC"
16:43:38 - No server with URI='urn:raspberrypi:UnifiedAutomation:UaServerCpp' has been discovered. The client could not discover a single server.
16:44:19 - Status: TibState.RunStopped
16:44:19 - Alarms: AlarmVector.NoAlarm
16:44:19 - Connection status: False TXclient_alive: False
16:44:19 - External delays: Local=3825.000 ns, Calibration=3825.000 ns, Sphe=3825.000 ns
16:44:19 - Trigger mask: 0x51 = TriggerMask.SlowControl|Software|Mono
16:44:19 - Input mask: 0x7
16:44:19 - Disable BUSY: False
16:44:19 - Extra BUSY Delay: 0.0
16:44:19 - Temperature: 32.12
16:44:25 - Starting task "Powering modules..."
16:44:25 - Executing command '/opt/cta/bin/cta_ctrl --slot 1 --all --power_on' on Digital Trigger Crate at 10.10.4.57
16:44:31 - Executing command '/opt/cta/bin/cta_ctrl --slot 5 --all --power_on' on Digital Trigger Crate at 10.10.4.57
16:44:36 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --all --power_on' on Digital Trigger Crate at 10.10.4.57
16:44:42 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --all --power_on' on Digital Trigger Crate at 10.10.4.57
16:44:48 - Executing command '/opt/cta/bin/cta_ctrl --slot 13 --all --power_on' on Digital Trigger Crate at 10.10.4.57
16:44:54 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --all --power_on' on Digital Trigger Crate at 10.10.4.57
16:44:59 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --all --power_on' on Digital Trigger Crate at 10.10.4.57
16:45:05 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --all --power_on' on Digital Trigger Crate at 10.10.4.57
16:45:11 - Finished task "Powering modules..."
16:45:15 - Scanning interface p4p1 for non configured modules...
16:45:16 - Found 61 MAC address(es)
16:45:24 - Starting task "Configure modules"
16:45:24 - Setting L0 thresholds to 15.0 p.e. per pixel using L0 slopes and offsets in /local/home/cta_test_bench/data/2019/20191203/l0_threshold_slope_and_offset_feb.h5
16:45:27 - Configuring modules with configuration /local/home/cta_test_bench/config/2020/20200625/config_2020-06-25_16-45-24.NMC.xml
16:45:27 - Switching off high voltages in configuration
16:45:31 - Successfully configured 53 module(s)
16:45:32 - All front-end boards are programmed with firmware F55D2001
16:45:33 - All backplanes are programmed with firmware 15
16:45:38 - NMC is in state NmcState.Configured and knows of 53 drawer(s)
16:45:39 - Found 0 PPS errors.
16:45:39 - Finished task "Configure modules"
16:45:54 - Calling method FSM.gotoObserving in object FSM
16:46:08 - Starting task "Switch HVPA voltage state"
16:46:08 - Setting HVPA current limit to 12489.75 uA
16:46:10 - High voltage locked channels will be checked every 30 s
16:46:14 - Stopping flashes...
16:46:20 - Starting NMC
16:46:20 - Clearing PPS error counters
16:46:26 - Setting HVPA ramp speed to 25 V/s
16:46:28 - Switching HVPA to state Nominal
16:46:37 - Setting HVPA current limits...
16:46:40 - Checking for high voltage locked pixels...
16:46:41 - Module 81 has 1 pixel(s) locked (HVPA status of 0x3f)!
16:46:42 - Unlocking 1 channel(s)...
16:46:42 - Resetting HVPA channel 6 of drawer 81...
16:46:51 - Calling method FSM.gotoReady in object FSM
16:47:00 - Finished task "Switch HVPA voltage state"
16:47:04 - Calling method FSM.gotoObserving in object FSM
16:47:10 - Checking for high voltage locked pixels...
16:47:11 - Module 146 has 1 pixel(s) locked (HVPA status of 0x7d)!
16:47:12 - Unlocking 1 channel(s)...
16:47:12 - Resetting HVPA channel 1 of drawer 146...
16:47:16 - Starting task "Check high voltages"
16:47:17 - Found 53 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
16:47:25 - Actual voltage is between 284.26 V and 1167.54 V.
16:47:26 - Low voltages range between 300.00 V and 300.00 V. 
16:47:35 - Reference voltages range between 945.26 V and 1167.15 V. 
16:47:40 - Checking for high voltage locked pixels...
16:47:44 - Maximum difference between expected and actual voltages is 11.33 V
16:47:48 - Maximum of current limits is 5647.07 uA
16:47:48 - Checking status of HVPA module in 53 drawer(s)...
16:47:49 - Checking status of HVPA channels in 53 drawer(s)...
16:47:53 - HV is locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
16:47:53 - Checking reference and low voltages against /local/home/cta_test_bench/config/2020/20200610/config_tower_53modules.NMC.xml
16:47:58 - Done
16:47:58 - Finished task "Check high voltages"
16:48:00 - Starting task "Check high voltages"
16:48:01 - Found 53 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
16:48:09 - Actual voltage is between 945.50 V and 1167.54 V.
16:48:10 - Low voltages range between 300.00 V and 300.00 V. 
16:48:10 - Checking for high voltage locked pixels...
16:48:18 - Configuring calibration box...
16:48:18 - Configuring calibration box with leds=0x40 voltage=9.00 frequency=999.0/3 central_current=0.0 duration=0 width=1 lemo_out=False fiber1_out=False fiber2_out=False lemo_in=False enabled=False
16:48:19 - Starting flashes...
16:48:19 - Reference voltages range between 945.26 V and 1167.15 V. 
16:48:23 - Checking UCTS clock...
16:48:24 - UCTS and control PC clocks differ by 2.072418 seconds
16:48:24 - Configuring UCTS
16:48:24 - Configuring TIB...
16:48:24 - Setting TIB forced mode to False
16:48:24 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
16:48:24 - Status: TibState.RunStopped
16:48:24 - Alarms: AlarmVector.NoAlarm
16:48:24 - Connection status: False TXclient_alive: False
16:48:24 - Disabling PPS
16:48:26 - Status: TibState.PpsStopped
16:48:26 - Alarms: AlarmVector.NoAlarm
16:48:26 - Connection status: False TXclient_alive: False
16:48:26 - Entering set_external_delay
16:48:26 - Setting ExternalDelayLocal to 0.000 ns
16:48:26 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
16:48:26 - Entering set_external_delay
16:48:26 - Setting ExternalDelayCalibration to 65.000 ns
16:48:26 - Writing 65.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
16:48:26 - Entering set_external_delay
16:48:26 - Setting ExternalDelaySphe to 150.000 ns
16:48:26 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
16:48:26 - Setting TIB trigger mode to 0x71 = TriggerMask.SlowControl|Pedestal|Software|Mono
16:48:26 - Writing 113 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
16:48:26 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v
16:48:26 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
16:48:26 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
16:48:26 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
16:48:26 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
16:48:26 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
16:48:26 - Status: TibState.PpsStopped
16:48:26 - Alarms: AlarmVector.NoAlarm
16:48:26 - Connection status: False TXclient_alive: False
16:48:26 - Writing True to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
16:48:26 - Writing 100 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
16:48:26 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:48:26 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
16:48:26 - Writing 100 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
16:48:26 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
16:48:27 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:48:27 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
16:48:27 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
16:48:27 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:48:28 - Maximum difference between expected and actual voltages is 0.70 V
16:48:29 - Connecting TIB to camera server at 10.10.4.12:222
16:48:29 - Starting TIB...
16:48:29 - Status: TibState.PpsStopped
16:48:29 - Alarms: AlarmVector.NoAlarm
16:48:29 - Connection status: False TXclient_alive: False
16:48:29 - Resetting run
16:48:29 - Wait for UCTS external trigger before enabling trigger...
16:48:29 - Status: TibState.Standby
16:48:29 - Alarms: AlarmVector.NoAlarm
16:48:29 - Connection status: False TXclient_alive: False
16:48:29 - Status: TibState.Standby
16:48:29 - Alarms: AlarmVector.NoAlarm
16:48:29 - Connection status: False TXclient_alive: False
16:48:30 - Status: TibState.Armed
16:48:30 - Alarms: AlarmVector.NoAlarm
16:48:30 - Connection status: True TXclient_alive: True
16:48:33 - Maximum of current limits is 5647.07 uA
16:48:33 - Checking status of HVPA module in 53 drawer(s)...
16:48:33 - Checking status of HVPA channels in 53 drawer(s)...
16:48:34 - Starting TIB triggers...
16:48:34 - Enabling trigger
16:48:34 - Status: TibState.Running
16:48:34 - Alarms: AlarmVector.NoAlarm
16:48:34 - Connection status: True TXclient_alive: True
16:48:37 - HV is stabilized in all channels.
16:48:37 - Checking reference and low voltages against /local/home/cta_test_bench/config/2020/20200610/config_tower_53modules.NMC.xml
16:48:40 - Checking for high voltage locked pixels...
16:48:42 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
16:48:42 - Alarms: AlarmVector.ReadError
16:48:42 - Temperature: 34.69
16:48:42 - Local camera rate: 334.0 Hz
16:48:42 - Stereo trigger rate: 0.0 Hz
16:48:42 - SPE trigger rate: 0.0 Hz
16:48:42 - Calibration trigger rate: 0.0 Hz
16:48:42 - Software trigger rate: 0.0 Hz
16:48:42 - Pedestal trigger rate: 100.0 Hz
16:48:42 - Collected trigger rate: 434.0 Hz
16:48:42 - Busy rate: 1.0 Hz
16:48:42 - Camera trigger rate: 433.0 Hz
16:48:43 - Done
16:48:43 - Finished task "Check high voltages"
16:48:56 - Stopping TIB...
16:48:56 - Status: TibState.Running
16:48:56 - Alarms: AlarmVector.ReadError
16:48:56 - Connection status: True TXclient_alive: True
16:48:56 - Disabling trigger
16:48:57 - Disabling PPS
16:48:58 - Status: TibState.PpsStopped
16:48:58 - Alarms: AlarmVector.ReadError
16:48:58 - Connection status: True TXclient_alive: True
16:48:58 - Closing TIB connection to camera server
16:48:59 - Status: TibState.PpsStopped
16:48:59 - Alarms: AlarmVector.ReadError
16:48:59 - Connection status: False TXclient_alive: False
16:48:59 - Run duration: 0:00:27
16:48:59 - Local camera triggers: 8827
16:48:59 - Slow control triggers: 0
16:48:59 - Single photo-electron triggers: 0
16:48:59 - Calibration triggers: 0
16:48:59 - Software triggers from UCTS: 0
16:48:59 - Pedestal triggers: 2648
16:48:59 - Collected input triggers: 9942
16:48:59 - Busy triggers: 10
16:48:59 - Camera event triggers sent to clusters: 9932
16:49:02 - Calling method FSM.gotoReady in object FSM