17:46:07 - Finished task "Powering modules..."
17:46:10 - Module 77 is connected to channel 1 of DTC slot 19
17:46:17 - Module 77 is connected to channel 1 of DTC slot 19
17:46:19 - Starting task "Powering modules..."
17:46:19 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 1 --power_on' on Digital Trigger Crate at 10.10.4.57
17:46:21 - Finished task "Powering modules..."
17:46:21 - Module 78 is connected to channel 8 of DTC slot 19
17:46:23 - Starting task "Powering modules..."
17:46:23 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 8 --power_on' on Digital Trigger Crate at 10.10.4.57
17:46:25 - Finished task "Powering modules..."
17:46:26 - Scanning interface enp2s0 for non configured modules...
17:46:27 - Found 1 MAC address(es)
17:46:29 - Scanning interface enp2s0 for non configured modules...
17:46:29 - Found 1 MAC address(es)
17:46:30 - Scanning interface enp2s0 for non configured modules...
17:46:31 - Found 2 MAC address(es)
17:49:37 - Module 78 is connected to channel 8 of DTC slot 19
17:49:39 - Starting task "Powering modules..."
17:49:39 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 8 --power_off' on Digital Trigger Crate at 10.10.4.57
17:49:41 - Finished task "Powering modules..."
17:50:37 - Starting task "Configure modules"
17:50:37 - Configuring modules with configuration /data/config/2020/20200924/config_tower_1module77_FEBv6_pingpong_Ibof_Dacl_DelayBusy0.NMC.xml
17:50:37 - Switching off high voltages in configuration
17:50:37 - Removing neighbouring module registers
17:50:40 - Successfully configured 1 module(s)
17:50:41 - All front-end boards are programmed with firmware F61D2009
17:50:41 - All backplanes are programmed with firmware 15
17:50:42 - NMC is in state NmcState.Configured and knows of 1 drawer(s)
17:50:42 - Found 0 PPS errors.
17:50:42 - Finished task "Configure modules"
17:50:43 - Starting task "Set L0 trigger thresholds"
17:50:43 - Setting L0 thresholds to 15.0 p.e. using /data/luts/l0_threshold_slope_and_offset_feb_20200708.h5
17:50:44 - Setting L0 thresholds to 15.0 p.e. per pixel using L0 slopes and offsets in /data/luts/l0_threshold_slope_and_offset_feb_20200708.h5
17:50:44 - Thresholds saved in configuration file /data/config/2020/20200924/config_2020-09-24_17-50-44.NMC.xml
17:50:44 - Finished task "Set L0 trigger thresholds"
17:50:57 - Starting task "Switch HVPA voltage state"
17:50:57 - Setting HVPA current limit to 12489.75 uA
17:50:57 - Setting HVPA ramp speed to 25 V/s
17:50:57 - Switching HVPA to state Nominal
17:51:00 - Resetting TIB...
17:51:04 - Setting HVPA current limits...
17:51:05 - Finished task "Switch HVPA voltage state"
17:51:09 - Configuring calibration box...
17:51:09 - Configuring calibration box with leds=0x1fff voltage=11.00 frequency=5000.0/1 central_current=0.0 duration=0 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=True enabled=True
17:51:31 - High voltage locked channels will be checked every 30 s
17:51:32 - Starting task "Check high voltages"
17:51:32 - Found 1 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
17:51:32 - Actual voltage is between 918.05 V and 931.47 V.
17:51:32 - Low voltages range between 300.00 V and 300.00 V. 
17:51:32 - Reference voltages range between 954.64 V and 1172.13 V. 
17:51:33 - Maximum difference between expected and actual voltages is 237.00 V
17:51:33 - Maximum of current limits is 5411.92 uA
17:51:33 - Checking status of HVPA module in 1 drawer(s)...
17:51:33 - Checking status of HVPA channels in 1 drawer(s)...
17:51:33 - HV is locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
17:51:33 - Checking reference and low voltages against /data/config/2020/20200924/config_tower_1module77_FEBv6_pingpong_Ibof_Dacl_DelayBusy0.NMC.xml
17:51:33 - Done
17:51:33 - Finished task "Check high voltages"
17:51:40 - Starting task "Check high voltages"
17:51:40 - Found 1 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
17:51:40 - Actual voltage is between 954.65 V and 1138.87 V.
17:51:40 - Low voltages range between 300.00 V and 300.00 V. 
17:51:41 - Reference voltages range between 954.64 V and 1172.13 V. 
17:51:41 - Maximum difference between expected and actual voltages is 19.23 V
17:51:41 - Maximum of current limits is 5411.92 uA
17:51:41 - Checking status of HVPA module in 1 drawer(s)...
17:51:41 - Checking status of HVPA channels in 1 drawer(s)...
17:51:41 - HV is locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
17:51:41 - Checking reference and low voltages against /data/config/2020/20200924/config_tower_1module77_FEBv6_pingpong_Ibof_Dacl_DelayBusy0.NMC.xml
17:51:41 - Done
17:51:41 - Finished task "Check high voltages"
17:51:47 - Starting task "Check high voltages"
17:51:47 - Found 1 drawer(s) in Nominal state, 0 in Low state and 0 in Wait state
17:51:47 - Actual voltage is between 954.65 V and 1172.42 V.
17:51:47 - Low voltages range between 300.00 V and 300.00 V. 
17:51:48 - Reference voltages range between 954.64 V and 1172.13 V. 
17:51:48 - Maximum difference between expected and actual voltages is 0.62 V
17:51:48 - Maximum of current limits is 5411.92 uA
17:51:48 - Checking status of HVPA module in 1 drawer(s)...
17:51:48 - Checking status of HVPA channels in 1 drawer(s)...
17:51:48 - HV is stabilized in all channels.
17:51:48 - Checking reference and low voltages against /data/config/2020/20200924/config_tower_1module77_FEBv6_pingpong_Ibof_Dacl_DelayBusy0.NMC.xml
17:51:48 - Done
17:51:49 - Finished task "Check high voltages"
17:51:50 - Updating L0 scaler widget...
17:51:53 - Updating L1 scaler widget...
17:52:02 - Checking for high voltage locked pixels...
17:52:02 - Starting NMC
17:52:02 - Clearing PPS error counters
17:52:12 - Calling method FSM.gotoObserving in object FSM
17:52:31 - Checking UCTS clock...
17:52:31 - UCTS clock is late by 6 seconds!
17:52:31 - Configuring UCTS
17:52:31 - Configuring TIB...
17:52:31 - Setting TIB forced mode to False
17:52:31 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
17:52:31 - Status: TibState.RunStopped
17:52:31 - Alarms: AlarmVector.NoAlarm
17:52:31 - Connection status: False TXclient_alive: False
17:52:31 - Disabling PPS
17:52:33 - Status: TibState.PpsStopped
17:52:33 - Alarms: AlarmVector.NoAlarm
17:52:33 - Connection status: False TXclient_alive: False
17:52:33 - Entering set_external_delay
17:52:33 - Setting ExternalDelayLocal to 0.000 ns
17:52:33 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
17:52:34 - Entering set_external_delay
17:52:34 - Setting ExternalDelayCalibration to 65.000 ns
17:52:34 - Writing 65.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
17:52:34 - Entering set_external_delay
17:52:34 - Setting ExternalDelaySphe to 150.000 ns
17:52:34 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
17:52:34 - Setting TIB trigger mode to 0x51 = TriggerMask.SlowControl|Software|Mono
17:52:34 - Writing 81 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
17:52:35 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v
17:52:35 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
17:52:35 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
17:52:35 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
17:52:35 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
17:52:35 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
17:52:35 - Status: TibState.PpsStopped
17:52:35 - Alarms: AlarmVector.NoAlarm
17:52:35 - Connection status: False TXclient_alive: False
17:52:35 - Writing False to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
17:52:35 - Writing 2000 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
17:52:35 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
17:52:35 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
17:52:35 - Writing 2000 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
17:52:35 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
17:52:35 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
17:52:35 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
17:52:36 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
17:52:36 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
17:52:36 - Checking for high voltage locked pixels...
17:52:37 - Connecting TIB to camera server at 10.10.4.12:222
17:52:37 - Starting TIB...
17:52:37 - Status: TibState.PpsStopped
17:52:37 - Alarms: AlarmVector.NoAlarm
17:52:37 - Connection status: False TXclient_alive: True
17:52:37 - Resetting run
17:52:37 - Wait for UCTS external trigger before enabling trigger...
17:52:37 - Status: TibState.Standby
17:52:37 - Alarms: AlarmVector.NoAlarm
17:52:37 - Connection status: False TXclient_alive: True
17:52:37 - Status: TibState.Standby
17:52:37 - Alarms: AlarmVector.NoAlarm
17:52:38 - Connection status: False TXclient_alive: True
17:52:39 - Status: TibState.Armed
17:52:39 - Alarms: AlarmVector.NoAlarm
17:52:39 - Connection status: True TXclient_alive: True
17:52:41 - Starting TIB triggers...
17:52:41 - Enabling trigger
17:52:41 - Status: TibState.Running
17:52:41 - Alarms: AlarmVector.NoAlarm
17:52:41 - Connection status: True TXclient_alive: True
17:52:42 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:42 - Alarms: AlarmVector.NoAlarm
17:52:42 - Temperature: 41.12
17:52:42 - Local camera rate: 7218.0 Hz
17:52:42 - Stereo trigger rate: 0.0 Hz
17:52:42 - SPE trigger rate: 0.0 Hz
17:52:42 - Calibration trigger rate: 7418.0 Hz
17:52:42 - Software trigger rate: 0.0 Hz
17:52:42 - Pedestal trigger rate: 0.0 Hz
17:52:42 - Collected trigger rate: 7173.0 Hz
17:52:42 - Busy rate: 6.0 Hz
17:52:42 - Camera trigger rate: 7167.0 Hz
17:52:44 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:44 - Alarms: AlarmVector.NoAlarm
17:52:44 - Temperature: 41.12
17:52:44 - Local camera rate: 7218.0 Hz
17:52:44 - Stereo trigger rate: 0.0 Hz
17:52:44 - SPE trigger rate: 0.0 Hz
17:52:44 - Calibration trigger rate: 7418.0 Hz
17:52:44 - Software trigger rate: 0.0 Hz
17:52:44 - Pedestal trigger rate: 0.0 Hz
17:52:44 - Collected trigger rate: 7173.0 Hz
17:52:44 - Busy rate: 6.0 Hz
17:52:44 - Camera trigger rate: 7167.0 Hz
17:52:45 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:45 - Alarms: AlarmVector.NoAlarm
17:52:45 - Temperature: 41.12
17:52:45 - Local camera rate: 7218.0 Hz
17:52:45 - Stereo trigger rate: 0.0 Hz
17:52:45 - SPE trigger rate: 0.0 Hz
17:52:45 - Calibration trigger rate: 7418.0 Hz
17:52:45 - Software trigger rate: 0.0 Hz
17:52:45 - Pedestal trigger rate: 0.0 Hz
17:52:45 - Collected trigger rate: 7173.0 Hz
17:52:45 - Busy rate: 6.0 Hz
17:52:45 - Camera trigger rate: 7167.0 Hz
17:52:47 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:47 - Alarms: AlarmVector.NoAlarm
17:52:47 - Temperature: 41.19
17:52:47 - Local camera rate: 7271.0 Hz
17:52:47 - Stereo trigger rate: 0.0 Hz
17:52:47 - SPE trigger rate: 0.0 Hz
17:52:47 - Calibration trigger rate: 7459.0 Hz
17:52:47 - Software trigger rate: 0.0 Hz
17:52:47 - Pedestal trigger rate: 0.0 Hz
17:52:47 - Collected trigger rate: 7271.0 Hz
17:52:47 - Busy rate: 7.0 Hz
17:52:47 - Camera trigger rate: 7264.0 Hz
17:52:48 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:48 - Alarms: AlarmVector.NoAlarm
17:52:48 - Temperature: 41.19
17:52:48 - Local camera rate: 7271.0 Hz
17:52:48 - Stereo trigger rate: 0.0 Hz
17:52:48 - SPE trigger rate: 0.0 Hz
17:52:48 - Calibration trigger rate: 7459.0 Hz
17:52:48 - Software trigger rate: 0.0 Hz
17:52:48 - Pedestal trigger rate: 0.0 Hz
17:52:48 - Collected trigger rate: 7271.0 Hz
17:52:48 - Busy rate: 7.0 Hz
17:52:48 - Camera trigger rate: 7264.0 Hz
17:52:49 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:49 - Alarms: AlarmVector.NoAlarm
17:52:49 - Temperature: 41.12
17:52:49 - Local camera rate: 7203.0 Hz
17:52:49 - Stereo trigger rate: 0.0 Hz
17:52:49 - SPE trigger rate: 0.0 Hz
17:52:49 - Calibration trigger rate: 7410.0 Hz
17:52:49 - Software trigger rate: 0.0 Hz
17:52:49 - Pedestal trigger rate: 0.0 Hz
17:52:49 - Collected trigger rate: 7203.0 Hz
17:52:49 - Busy rate: 2.0 Hz
17:52:49 - Camera trigger rate: 7201.0 Hz
17:52:51 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:51 - Alarms: AlarmVector.ReadError
17:52:51 - Temperature: 41.19
17:52:51 - Local camera rate: 7203.0 Hz
17:52:51 - Stereo trigger rate: 0.0 Hz
17:52:51 - SPE trigger rate: 0.0 Hz
17:52:51 - Calibration trigger rate: 7410.0 Hz
17:52:51 - Software trigger rate: 0.0 Hz
17:52:51 - Pedestal trigger rate: 0.0 Hz
17:52:51 - Collected trigger rate: 7203.0 Hz
17:52:51 - Busy rate: 2.0 Hz
17:52:51 - Camera trigger rate: 7201.0 Hz
17:52:52 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:52 - Alarms: AlarmVector.ReadError
17:52:52 - Temperature: 41.19
17:52:52 - Local camera rate: 7111.0 Hz
17:52:52 - Stereo trigger rate: 0.0 Hz
17:52:52 - SPE trigger rate: 0.0 Hz
17:52:52 - Calibration trigger rate: 7294.0 Hz
17:52:52 - Software trigger rate: 0.0 Hz
17:52:52 - Pedestal trigger rate: 0.0 Hz
17:52:52 - Collected trigger rate: 7111.0 Hz
17:52:52 - Busy rate: 7111.0 Hz
17:52:52 - Camera trigger rate: 0.0 Hz
17:52:54 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:54 - Alarms: AlarmVector.ReadError
17:52:54 - Temperature: 41.12
17:52:54 - Local camera rate: 7117.0 Hz
17:52:54 - Stereo trigger rate: 0.0 Hz
17:52:54 - SPE trigger rate: 0.0 Hz
17:52:54 - Calibration trigger rate: 7299.0 Hz
17:52:54 - Software trigger rate: 0.0 Hz
17:52:54 - Pedestal trigger rate: 0.0 Hz
17:52:54 - Collected trigger rate: 7117.0 Hz
17:52:54 - Busy rate: 7117.0 Hz
17:52:54 - Camera trigger rate: 0.0 Hz
17:52:56 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
17:52:56 - Alarms: AlarmVector.ReadError
17:52:56 - Temperature: 41.12
17:52:56 - Local camera rate: 7117.0 Hz
17:52:56 - Stereo trigger rate: 0.0 Hz
17:52:56 - SPE trigger rate: 0.0 Hz
17:52:56 - Calibration trigger rate: 7299.0 Hz
17:52:56 - Software trigger rate: 0.0 Hz
17:52:56 - Pedestal trigger rate: 0.0 Hz
17:52:56 - Collected trigger rate: 7117.0 Hz
17:52:56 - Busy rate: 7117.0 Hz
17:52:56 - Camera trigger rate: 0.0 Hz
17:52:58 - Stopping TIB...
17:52:58 - Status: TibState.Running
17:52:58 - Alarms: AlarmVector.ReadError
17:52:58 - Connection status: True TXclient_alive: True
17:52:58 - Disabling trigger
17:52:58 - Disabling PPS
17:52:59 - Status: TibState.PpsStopped
17:52:59 - Alarms: AlarmVector.ReadError
17:52:59 - Connection status: True TXclient_alive: True
17:52:59 - Closing TIB connection to camera server
17:53:00 - Status: TibState.PpsStopped
17:53:00 - Alarms: AlarmVector.ReadError
17:53:00 - Connection status: False TXclient_alive: False
17:53:00 - Run duration: 0:00:19
17:53:00 - Local camera triggers: 134000
17:53:00 - Slow control triggers: 0
17:53:00 - Single photo-electron triggers: 0
17:53:00 - Calibration triggers: 137339
17:53:00 - Software triggers from UCTS: 0
17:53:00 - Pedestal triggers: 0
17:53:00 - Collected input triggers: 118632
17:53:00 - Busy triggers: 57245
17:53:00 - Camera event triggers sent to clusters: 61387
17:53:02 - Checking for high voltage locked pixels...
17:53:04 - Calling method FSM.gotoReady in object FSM