14:17:03 - Module 84 is connected to channel 12 of DTC slot 19
14:17:06 - Starting task "Powering modules..."
14:17:06 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 12 --power_on' on Digital Trigger Crate at 10.10.4.57
14:17:08 - Finished task "Powering modules..."
14:17:12 - Scanning interface enp2s0 for non configured modules...
14:17:14 - Found 1 MAC address(es)
14:17:22 - Starting task "Configure modules"
14:17:22 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.5.1-40061 
14:17:22 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.5.1-40061 
14:17:22 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.5.1-40061 
14:17:22 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.5.1-40061
14:17:22 - Configuring modules with configuration /data/config/2020/20201116/config_tower_1module84_FEBv6_nopingpong_Dacl_DelayBusy7.NMC.xml
14:17:22 - Switching off high voltages in configuration
14:17:26 - Successfully configured 1 module(s)
14:17:26 - All front-end boards are programmed with firmware F61D2010
14:17:26 - All backplanes are programmed with firmware 15
14:17:27 - NMC is in state NmcState.Configured and knows of 1 drawer(s)
14:17:27 - Found 0 PPS errors.
14:17:27 - Finished task "Configure modules"
14:44:02 - Resetting NMC
14:44:05 - Starting task "Configure modules"
14:44:05 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.5.1-40061 
14:44:05 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.5.1-40061 
14:44:05 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.5.1-40061 
14:44:05 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.5.1-40061
14:44:05 - Configuring modules with configuration /data/config/2020/20201116/config_tower_1module84_FEBv6_nopingpong_Dacl_DelayBusy7.NMC.xml
14:44:05 - Switching off high voltages in configuration
14:44:05 - ("NmcError: Failed to call 'Configure': status Bad statuses are present, code 0xfffffffe, status NmcStatus.Timeout", 4294967294)
14:44:05 - NMC is in state NmcState.Ready and knows of 0 drawer(s)
14:44:05 - Found 0 PPS errors.
14:44:06 - Finished task "Configure modules"
14:44:12 - Starting task "Powering modules..."
14:44:12 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 12 --power_off' on Digital Trigger Crate at 10.10.4.57
14:44:13 - Finished task "Powering modules..."
14:44:16 - Starting task "Powering modules..."
14:44:16 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 12 --power_on' on Digital Trigger Crate at 10.10.4.57
14:44:17 - Finished task "Powering modules..."
14:44:21 - Starting task "Configure modules"
14:44:21 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.5.1-40061 
14:44:21 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.5.1-40061 
14:44:21 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.5.1-40061 
14:44:21 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.5.1-40061
14:44:21 - Configuring modules with configuration /data/config/2020/20201116/config_tower_1module84_FEBv6_nopingpong_Dacl_DelayBusy7.NMC.xml
14:44:21 - Switching off high voltages in configuration
14:44:22 - ("NmcError: Failed to call 'Configure': status Bad statuses are present, code 0xfffffffe, status NmcStatus.Timeout", 4294967294)
14:44:22 - NMC is in state NmcState.Ready and knows of 0 drawer(s)
14:44:22 - Found 0 PPS errors.
14:44:22 - Finished task "Configure modules"
14:44:39 - Starting task "Powering modules..."
14:44:39 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 12 --power_off' on Digital Trigger Crate at 10.10.4.57
14:44:41 - Finished task "Powering modules..."
14:44:48 - Starting task "Powering modules..."
14:44:48 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 12 --power_on' on Digital Trigger Crate at 10.10.4.57
14:44:50 - Finished task "Powering modules..."
14:44:53 - Scanning interface enp2s0 for non configured modules...
14:44:56 - Found 1 MAC address(es)
14:45:01 - Starting task "Configure modules"
14:45:01 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.5.1-40061 
14:45:01 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.5.1-40061 
14:45:01 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.5.1-40061 
14:45:01 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.5.1-40061
14:45:01 - Configuring modules with configuration /data/config/2020/20201116/config_tower_1module84_FEBv6_nopingpong_Dacl_DelayBusy7.NMC.xml
14:45:01 - Switching off high voltages in configuration
14:45:05 - Successfully configured 1 module(s)
14:45:05 - All front-end boards are programmed with firmware 61201117
14:45:06 - All backplanes are programmed with firmware 15
14:45:06 - NMC is in state NmcState.Configured and knows of 1 drawer(s)
14:45:06 - Found 0 PPS errors.
14:45:06 - Finished task "Configure modules"
14:45:44 - Starting NMC
14:45:44 - Clearing PPS error counters
14:46:05 - Calling method FSM.gotoReady in object FSM
14:46:53 - Calling method FSM.gotoReady in object FSM
14:47:01 - Calling method FSM.gotoReady in object FSM
14:47:01 - Failed to call FSM.gotoReady: Bad statuses are present
14:47:51 - Calling method FSM.gotoReady in object FSM
14:47:57 - Calling method FSM.gotoObserving in object FSM
14:48:37 - Starting task "Powering modules..."
14:48:37 - Executing command '/opt/cta/bin/cta_ctrl --slot 1 --all --power_on' on Digital Trigger Crate at 10.10.4.57
14:48:42 - Executing command '/opt/cta/bin/cta_ctrl --slot 5 --all --power_on' on Digital Trigger Crate at 10.10.4.57
14:48:46 - Scanning interface enp2s0 for non configured modules...
14:48:51 - Found 1 MAC address(es)
14:48:48 - Executing command '/opt/cta/bin/cta_ctrl --slot 6 --all --power_on' on Digital Trigger Crate at 10.10.4.57
14:48:54 - Executing command '/opt/cta/bin/cta_ctrl --slot 9 --all --power_on' on Digital Trigger Crate at 10.10.4.57
14:49:00 - Executing command '/opt/cta/bin/cta_ctrl --slot 13 --all --power_on' on Digital Trigger Crate at 10.10.4.57
14:49:06 - Executing command '/opt/cta/bin/cta_ctrl --slot 14 --all --power_on' on Digital Trigger Crate at 10.10.4.57
14:49:10 - Scanning interface enp2s0 for non configured modules...
14:49:11 - Found 13 MAC address(es)
14:49:11 - Executing command '/opt/cta/bin/cta_ctrl --slot 15 --all --power_on' on Digital Trigger Crate at 10.10.4.57
14:49:13 - Scanning interface enp2s0 for non configured modules...
14:49:14 - Found 17 MAC address(es)
14:49:15 - Scanning interface enp2s0 for non configured modules...
14:49:16 - Found 17 MAC address(es)
14:49:16 - Scanning interface enp2s0 for non configured modules...
14:49:17 - Found 17 MAC address(es)
14:49:17 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --all --power_on' on Digital Trigger Crate at 10.10.4.57
14:49:18 - Scanning interface enp2s0 for non configured modules...
14:49:19 - Found 17 MAC address(es)
14:49:20 - Scanning interface enp2s0 for non configured modules...
14:49:20 - Found 19 MAC address(es)
14:49:21 - Scanning interface enp2s0 for non configured modules...
14:49:22 - Found 19 MAC address(es)
14:49:23 - Finished task "Powering modules..."
14:49:25 - Scanning interface enp2s0 for non configured modules...
14:49:25 - Found 22 MAC address(es)
14:49:26 - Scanning interface enp2s0 for non configured modules...
14:49:27 - Found 26 MAC address(es)
14:49:27 - Scanning interface enp2s0 for non configured modules...
14:49:28 - Found 26 MAC address(es)
14:49:29 - Scanning interface enp2s0 for non configured modules...
14:49:29 - Found 26 MAC address(es)
14:49:30 - Scanning interface enp2s0 for non configured modules...
14:49:30 - Found 26 MAC address(es)
14:49:32 - Scanning interface enp2s0 for non configured modules...
14:49:33 - Found 26 MAC address(es)
14:49:33 - Scanning interface enp2s0 for non configured modules...
14:49:34 - Found 26 MAC address(es)
14:50:30 - Calling method FSM.gotoReady in object FSM
15:01:44 - Starting task "Powering modules..."
15:01:44 - Executing command '/opt/cta/bin/cta_ctrl --all --power_off' on Digital Trigger Crate at 10.10.4.57
15:01:45 - Finished task "Powering modules..."
15:01:48 - Stopping NMC
15:01:48 - Found 0 PPS errors.
15:01:52 - Resetting NMC
15:03:45 - Calling method FSM.gotoReady in object FSM
15:04:00 - Calling method FSM.gotoObserving in object FSM
15:05:09 - Module 84 is connected to channel 12 of DTC slot 19
15:05:11 - Starting task "Powering modules..."
15:05:11 - Executing command '/opt/cta/bin/cta_ctrl --slot 19 --channel 12 --power_on' on Digital Trigger Crate at 10.10.4.57
15:05:13 - Finished task "Powering modules..."
15:05:15 - ECC is in state EccState.Ready with status EccStatus.NoError
15:05:15 - Average temperature: 20.3 deg C
15:05:15 - Air temperature (T1-T6 and T9-T10): 20.3, -838.0, 1254.0, -838.0, 1254.0, 1254.0, 1254.0, 1253.0 deg C
15:05:15 - Air differential pressure: 0.0 Pa
15:05:15 - Water temperature (T11-T12): -838.0, 1254.0 deg C
15:05:15 - Back doors are closed.
15:05:15 - Shutter is in state PartiallyOpen (error 0)
15:05:15 - Light sensors: [1.63, 1.95, 2.25, 2.51] V
15:05:15 - Humidity: 0.0 %. Water pressure: [0.0, 0.0] bar.
15:05:15 - Fan speeds: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0] rpm
15:05:15 - 24 V power supply no. 1 is enabled with a current of 0.530 A.
15:05:16 - ECC is in state EccState.Ready with status EccStatus.NoError
15:05:16 - Average temperature: 20.3 deg C
15:05:16 - Air temperature (T1-T6 and T9-T10): 20.3, -838.0, 1254.0, -838.0, 1254.0, 1254.0, 1254.0, 1253.0 deg C
15:05:16 - Air differential pressure: 0.0 Pa
15:05:16 - Water temperature (T11-T12): -838.0, 1254.0 deg C
15:05:16 - Back doors are closed.
15:05:16 - Shutter is in state PartiallyOpen (error 0)
15:05:16 - Light sensors: [1.64, 1.96, 2.25, 2.52] V
15:05:16 - Humidity: 0.0 %. Water pressure: [0.0, 0.0] bar.
15:05:17 - Fan speeds: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0] rpm
15:05:17 - 24 V power supply no. 1 is enabled with a current of 0.530 A.
15:05:18 - ECC is in state EccState.Ready with status EccStatus.NoError
15:05:18 - Average temperature: 20.3 deg C
15:05:18 - Air temperature (T1-T6 and T9-T10): 20.3, 1255.0, 1254.0, -838.0, 1254.0, 1254.0, 1254.0, -838.0 deg C
15:05:18 - Air differential pressure: 0.0 Pa
15:05:18 - Water temperature (T11-T12): 1253.0, 1254.0 deg C
15:05:18 - Back doors are closed.
15:05:18 - Shutter is in state PartiallyOpen (error 0)
15:05:18 - Light sensors: [1.64, 1.96, 2.25, 2.52] V
15:05:18 - Humidity: 0.0 %. Water pressure: [0.0, 0.0] bar.
15:05:18 - Fan speeds: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0] rpm
15:05:18 - 24 V power supply no. 1 is enabled with a current of 0.530 A.
15:05:19 - ECC is in state EccState.Ready with status EccStatus.NoError
15:05:19 - Average temperature: 20.3 deg C
15:05:19 - Air temperature (T1-T6 and T9-T10): 20.3, -838.0, -838.0, 1254.0, -838.0, 1254.0, 1254.0, 1253.0 deg C
15:05:19 - Air differential pressure: 0.0 Pa
15:05:19 - Water temperature (T11-T12): 1253.0, 1254.0 deg C
15:05:19 - Back doors are closed.
15:05:19 - Shutter is in state PartiallyOpen (error 0)
15:05:19 - Light sensors: [1.64, 1.96, 2.25, 2.52] V
15:05:19 - Humidity: 0.0 %. Water pressure: [0.0, 0.0] bar.
15:05:19 - Fan speeds: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0] rpm
15:05:19 - 24 V power supply no. 1 is enabled with a current of 0.950 A.
15:05:22 - Scanning interface enp2s0 for non configured modules...
15:05:23 - Found 1 MAC address(es)
15:05:31 - Starting task "Configure modules"
15:05:31 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.5.1-40061 
15:05:31 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.5.1-40061 
15:05:31 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.5.1-40061 
15:05:31 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.5.1-40061
15:05:31 - Configuring modules with configuration /data/config/2020/20201116/config_tower_1module84_FEBv6_nopingpong_Dacl_DelayBusy7.NMC.xml
15:05:31 - Switching off high voltages in configuration
15:05:35 - Successfully configured 1 module(s)
15:05:35 - All front-end boards are programmed with firmware 61201117
15:05:35 - All backplanes are programmed with firmware 15
15:05:36 - NMC is in state NmcState.Configured and knows of 1 drawer(s)
15:05:36 - Found 0 PPS errors.
15:05:36 - Finished task "Configure modules"
15:05:40 - Starting NMC
15:05:40 - Clearing PPS error counters
15:06:23 - Checking UCTS clock...
15:06:24 - UCTS clock is late by 21 seconds!
15:06:24 - Configuring UCTS
15:06:24 - Configuring TIB...
15:06:24 - Setting TIB forced mode to False
15:06:24 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
15:06:24 - Status: TibState.RunStopped
15:06:24 - Alarms: AlarmVector.BufferFull
15:06:24 - Connection status: False TXclient_alive: False
15:06:24 - Disabling PPS
15:06:26 - Status: TibState.PpsStopped
15:06:27 - Alarms: AlarmVector.BufferFull
15:06:27 - Connection status: False TXclient_alive: False
15:06:27 - Entering set_external_delay
15:06:27 - Setting ExternalDelayLocal to 0.000 ns
15:06:27 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
15:06:27 - Entering set_external_delay
15:06:27 - Setting ExternalDelayCalibration to 65.000 ns
15:06:27 - Writing 65.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
15:06:27 - Entering set_external_delay
15:06:27 - Setting ExternalDelaySphe to 150.000 ns
15:06:27 - Writing 150.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
15:06:27 - Setting TIB trigger mode to 0x70 = TriggerMask.SlowControl|Pedestal|Software
15:06:27 - Writing 112 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
15:06:27 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v
15:06:27 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
15:06:27 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
15:06:27 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
15:06:27 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
15:06:27 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
15:06:27 - Status: TibState.PpsStopped
15:06:27 - Alarms: AlarmVector.BufferFull
15:06:27 - Connection status: False TXclient_alive: False
15:06:27 - Writing True to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
15:06:27 - Writing 1000 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
15:06:27 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:06:27 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
15:06:27 - Writing 1000 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
15:06:27 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
15:06:27 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:06:27 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
15:06:27 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
15:06:27 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:06:39 - Starting task "Set DTC trigger masks"
15:06:39 - Configuring DTC trigger masks for 1 configured module(s)
15:06:39 - Setting DTC slot 1 trigger mask to 0x0000
15:06:43 - Setting DTC slot 5 trigger mask to 0x0000
15:06:46 - Setting DTC slot 6 trigger mask to 0x0000
15:06:50 - Setting DTC slot 9 trigger mask to 0x0000
15:06:53 - Setting DTC slot 13 trigger mask to 0x0000
15:06:57 - Setting DTC slot 14 trigger mask to 0x0000
15:07:01 - Setting DTC slot 15 trigger mask to 0x0000
15:07:04 - Setting DTC slot 19 trigger mask to 0x1000
15:07:08 - Finished task "Set DTC trigger masks"
15:07:12 - Connecting TIB to camera server at 10.10.4.12:222
15:07:12 - Starting TIB...
15:07:12 - Status: TibState.PpsStopped
15:07:12 - Alarms: AlarmVector.BufferFull
15:07:12 - Connection status: False TXclient_alive: False
15:07:12 - Resetting run
15:07:12 - Wait for UCTS external trigger before enabling trigger...
15:07:12 - Status: TibState.Standby
15:07:12 - Alarms: AlarmVector.NoAlarm
15:07:12 - Connection status: False TXclient_alive: True
15:07:12 - Status: TibState.Standby
15:07:12 - Alarms: AlarmVector.NoAlarm
15:07:12 - Connection status: False TXclient_alive: True
15:07:13 - Status: TibState.Armed
15:07:13 - Alarms: AlarmVector.NoAlarm
15:07:13 - Connection status: True TXclient_alive: True
15:07:16 - Starting TIB triggers...
15:07:16 - Enabling trigger
15:07:16 - Status: TibState.Running
15:07:16 - Alarms: AlarmVector.ReadError
15:07:16 - Connection status: True TXclient_alive: True
15:07:18 - Status: TibState.Running ConnectionStatus: True TXclient_alive: True
15:07:18 - Alarms: AlarmVector.ReadError
15:07:18 - Temperature: 34.94
15:07:18 - Local camera rate: 0.0 Hz
15:07:18 - Stereo trigger rate: 0.0 Hz
15:07:18 - SPE trigger rate: 0.0 Hz
15:07:18 - Calibration trigger rate: 0.0 Hz
15:07:18 - Software trigger rate: 0.0 Hz
15:07:18 - Pedestal trigger rate: 1000.0 Hz
15:07:18 - Collected trigger rate: 1000.0 Hz
15:07:18 - Busy rate: 0.0 Hz
15:07:18 - Camera trigger rate: 1000.0 Hz
15:08:06 - Stopping TIB...
15:08:06 - Status: TibState.Running
15:08:06 - Alarms: AlarmVector.ReadError
15:08:06 - Connection status: True TXclient_alive: True
15:08:06 - Disabling trigger
15:08:06 - Disabling PPS
15:08:07 - Status: TibState.PpsStopped
15:08:07 - Alarms: AlarmVector.ReadError
15:08:07 - Connection status: True TXclient_alive: True
15:08:07 - Closing TIB connection to camera server
15:08:09 - Status: TibState.PpsStopped
15:08:09 - Alarms: AlarmVector.ReadError
15:08:09 - Connection status: False TXclient_alive: False
15:08:09 - Run duration: 0:00:53
15:08:10 - Local camera triggers: 0
15:08:10 - Slow control triggers: 0
15:08:10 - Single photo-electron triggers: 0
15:08:10 - Calibration triggers: 0
15:08:10 - Software triggers from UCTS: 0
15:08:10 - Pedestal triggers: 52410
15:08:10 - Collected input triggers: 50089
15:08:10 - Busy triggers: 0
15:08:10 - Camera event triggers sent to clusters: 50089
15:08:13 - Calling method FSM.gotoReady in object FSM
15:08:16 - Stopping NMC
15:08:16 - Found 0 PPS errors.