12:41:39 - Starting task "Configure modules" 12:41:39 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/utils/branches/v5.6.0-41551 12:41:39 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libNectarCam/branches/v5.6.0-41551 12:41:39 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/libQNectarCam/branches/v5.6.0-41551 12:41:39 - Version of NMC: svn+ssh://svn.in2p3.fr/cta/MST/NectarCAM/qNectarcamModuleController/branches/v5.6.0-41551 12:41:39 - Configuring modules with configuration /data/config/2021/20211011/config_cameraqm_265febs_newHV6.NMC.xml 12:41:39 - Switching off high voltages in configuration 12:41:50 - Successfully configured 265 module(s) 12:41:50 - Waiting for modules to be configured... 12:41:56 - All front-end boards are programmed with firmware F56D2002 12:42:09 - Found 2 different versions of DTBP firmware: [15 17] 12:43:09 - NMC is in state NmcState.Ready and knows of 265 drawer(s) 12:43:31 - All front-end boards have counted 0 event(s) 12:43:39 - Found 0 PPS errors. 12:43:39 - Finished task "Configure modules" 12:43:41 - Starting task "Switch HVPA voltage state" 12:43:41 - Setting HVPA current limit to 12489.75 uA 12:43:41 - Setting Hvpa.Hvpa0.CurrentTrip in all drawers 12:43:56 - Setting Hvpa.Hvpa1.CurrentTrip in all drawers 12:44:12 - Setting Hvpa.Hvpa2.CurrentTrip in all drawers 12:44:27 - Setting Hvpa.Hvpa3.CurrentTrip in all drawers 12:44:42 - Setting Hvpa.Hvpa4.CurrentTrip in all drawers 12:44:57 - Setting Hvpa.Hvpa5.CurrentTrip in all drawers 12:45:12 - Setting Hvpa.Hvpa6.CurrentTrip in all drawers 12:45:28 - Setting HVPA ramp speed to 25 V/s 12:45:45 - Switching HVPA to state Nominal 12:46:17 - Setting HVPA current limits... 12:48:16 - Finished task "Switch HVPA voltage state" 12:51:06 - High voltage locked channels will be checked every 60 s 12:52:06 - Checking for high voltage locked pixels... 12:52:31 - HVPA channel 2 of drawer 86 has status 0x0007! 12:52:31 - HVPA channel 2 in module 86 is in rise default! 12:52:54 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 1 channel(s) 12:53:06 - Checking for high voltage locked pixels... 12:53:31 - HVPA channel 2 of drawer 86 has status 0x0007! 12:53:31 - HVPA channel 2 in module 86 is in rise default! 12:53:54 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 1 channel(s) 12:54:06 - Checking for high voltage locked pixels... 12:54:31 - HVPA channel 2 of drawer 86 has status 0x0007! 12:54:31 - HVPA channel 2 in module 86 is in rise default! 12:54:54 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 1 channel(s) 12:55:06 - Checking for high voltage locked pixels... 12:55:31 - HVPA channel 2 of drawer 86 has status 0x0007! 12:55:31 - HVPA channel 2 in module 86 is in rise default! 12:55:54 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 1 channel(s) 12:56:06 - Checking for high voltage locked pixels... 12:56:31 - HVPA channel 2 of drawer 86 has status 0x0007! 12:56:31 - HVPA channel 2 in module 86 is in rise default! 12:56:48 - HVPA channel 5 in module 171 is still in progress 12:56:54 - HV is locked in 0 channel(s), was in rise default in 1 channel(s) and was still in progress in 2 channel(s) 12:56:54 - Stopped periodic check of locked high voltages 12:59:06 - Starting NMC 12:59:06 - Clearing PPS error counters 12:59:28 - Resetting FEB counters... 12:59:34 - Starting all configured modules... 13:01:14 - Calling method FSM.gotoObserving in object FSM 13:01:14 - Starting run 2683 13:01:24 - Checking UCTS clock... 13:01:24 - UCTS clock is late by 11 seconds! 13:01:24 - Configuring UCTS with (10.200.7.250, 10.200.4.100, 44:a8:42:44:32:c9) 13:01:25 - Configuring TIB... 13:01:25 - Setting TIB forced mode to False 13:01:25 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v 13:01:25 - Status: TibState.PpsStopped 13:01:25 - Alarms: AlarmVector.ReadError 13:01:25 - Connection status: False TXclient_alive: False 13:01:25 - Entering set_external_delay 13:01:25 - Setting ExternalDelayLocal to 27.000 ns 13:01:25 - Writing 27.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v 13:01:25 - Entering set_external_delay 13:01:25 - Setting ExternalDelayCalibration to 58.000 ns 13:01:25 - Writing 58.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v 13:01:25 - Entering set_external_delay 13:01:25 - Setting ExternalDelaySphe to 110.000 ns 13:01:25 - Writing 110.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v 13:01:25 - Setting TIB trigger mode to 0x54 = TriggerMask.SlowControl|Software|Calibration 13:01:25 - Writing 84 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v 13:01:25 - Writing 1 to Camera.TIB.InputsEnable.InputMask.InputMask_v 13:01:25 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v 13:01:25 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v 13:01:25 - Writing True to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v 13:01:25 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v 13:01:25 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v 13:01:25 - Status: TibState.PpsStopped 13:01:25 - Alarms: AlarmVector.ReadError 13:01:25 - Connection status: False TXclient_alive: False 13:01:25 - Writing False to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v 13:01:25 - Writing 100 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v 13:01:25 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 13:01:25 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v 13:01:25 - Writing 100 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v 13:01:25 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v 13:01:25 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 13:01:25 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v 13:01:25 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v 13:01:25 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v 13:01:26 - Connecting TIB to camera server at 10.10.4.12:222 13:01:26 - Starting TIB... 13:01:26 - Status: TibState.PpsStopped 13:01:26 - Alarms: AlarmVector.ReadError 13:01:26 - Connection status: False TXclient_alive: True 13:01:26 - Resetting run 13:01:26 - Wait for UCTS external trigger before enabling trigger... 13:01:26 - Status: TibState.Standby 13:01:26 - Alarms: AlarmVector.NoAlarm 13:01:27 - Connection status: False TXclient_alive: True 13:01:27 - Status: TibState.Standby 13:01:27 - Alarms: AlarmVector.NoAlarm 13:01:27 - Connection status: False TXclient_alive: True 13:01:28 - Status: TibState.Armed 13:01:28 - Alarms: AlarmVector.NoAlarm 13:01:28 - Connection status: True TXclient_alive: True 13:01:29 - Starting TIB triggers... 13:01:29 - Enabling trigger 13:01:29 - Status: TibState.Running 13:01:29 - Alarms: AlarmVector.NoAlarm 13:01:29 - Connection status: True TXclient_alive: True 13:01:36 - Configuring calibration box... 13:01:36 - Configuring calibration box with leds=0x40 voltage=10.98 frequency=1000.0/1 central_current=0.0 duration=0 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=False enabled=False 13:01:39 - Scanning voltages 13:01:39 - Starting task "Scan voltages" 13:01:39 - Setting LED mask to 0x0 13:01:39 - Configuring calibration box with leds=0x0 voltage=8.00 frequency=1000.0/1 central_current=0.0 duration=0 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=False enabled=False 13:01:39 - Starting flashes... 13:01:45 - Stopping flashes... 13:01:46 - Setting LED mask to 0x40 13:01:57 - Setting voltage to 8.000000 V 13:01:58 - Starting flashes... 13:02:04 - Stopping flashes... 13:02:15 - Setting voltage to 8.250000 V 13:02:16 - Starting flashes... 13:02:22 - Stopping flashes... 13:02:33 - Setting voltage to 8.500000 V 13:02:35 - Starting flashes... 13:02:41 - Stopping flashes... 13:02:52 - Setting voltage to 8.750000 V 13:02:53 - Starting flashes... 13:02:59 - Stopping flashes... 13:03:10 - Setting voltage to 9.000000 V 13:03:11 - Starting flashes... 13:03:17 - Stopping flashes... 13:03:28 - Setting voltage to 9.250000 V 13:03:29 - Starting flashes... 13:03:35 - Stopping flashes... 13:03:46 - Setting voltage to 9.500000 V 13:03:48 - Starting flashes... 13:03:54 - Stopping flashes... 13:04:05 - Setting voltage to 9.750000 V 13:04:06 - Starting flashes... 13:04:12 - Stopping flashes... 13:04:23 - Setting voltage to 10.000000 V 13:04:24 - Starting flashes... 13:04:30 - Stopping flashes... 13:04:41 - Setting voltage to 10.250000 V 13:04:42 - Starting flashes... 13:04:48 - Stopping flashes... 13:04:59 - Setting voltage to 10.500000 V 13:05:01 - Starting flashes... 13:05:07 - Stopping flashes... 13:05:18 - Setting voltage to 10.750000 V 13:05:19 - Starting flashes... 13:05:25 - Stopping flashes... 13:05:36 - Setting voltage to 11.000000 V 13:05:37 - Starting flashes... 13:05:43 - Stopping flashes... 13:05:54 - Setting voltage to 11.250000 V 13:05:55 - Starting flashes... 13:06:01 - Stopping flashes... 13:06:12 - Setting voltage to 11.500000 V 13:06:14 - Starting flashes... 13:06:20 - Stopping flashes... 13:06:31 - Setting voltage to 11.750000 V 13:06:32 - Starting flashes... 13:06:38 - Stopping flashes... 13:06:49 - Setting voltage to 12.000000 V 13:06:50 - Starting flashes... 13:06:56 - Stopping flashes... 13:07:07 - Setting voltage to 12.250000 V 13:07:08 - Starting flashes... 13:07:14 - Stopping flashes... 13:07:25 - Setting voltage to 12.500000 V 13:07:27 - Starting flashes... 13:07:33 - Stopping flashes... 13:07:44 - Setting voltage to 12.750000 V 13:07:45 - Starting flashes... 13:07:51 - Stopping flashes... 13:08:02 - Setting voltage to 13.000000 V 13:08:03 - Starting flashes... 13:08:09 - Stopping flashes... 13:08:20 - Setting voltage to 13.250000 V 13:08:21 - Starting flashes... 13:08:27 - Stopping flashes... 13:08:38 - Setting voltage to 13.500000 V 13:08:40 - Starting flashes... 13:08:46 - Stopping flashes... 13:08:57 - Setting voltage to 13.750000 V 13:08:58 - Starting flashes... 13:09:04 - Stopping flashes... 13:09:15 - Setting voltage to 14.000000 V 13:09:16 - Starting flashes... 13:09:22 - Stopping flashes... 13:09:33 - Setting voltage to 14.250000 V 13:09:34 - Starting flashes... 13:09:40 - Stopping flashes... 13:09:51 - Setting voltage to 14.500000 V 13:09:53 - Starting flashes... 13:09:59 - Stopping flashes... 13:10:10 - Setting voltage to 14.750000 V 13:10:11 - Starting flashes... 13:10:17 - Stopping flashes... 13:10:28 - Setting voltage to 15.000000 V 13:10:29 - Starting flashes... 13:10:35 - Stopping flashes... 13:10:46 - Setting voltage to 15.250000 V 13:10:47 - Starting flashes... 13:10:53 - Stopping flashes... 13:11:04 - Setting voltage to 15.500000 V 13:11:06 - Starting flashes... 13:11:12 - Stopping flashes... 13:11:23 - Setting voltage to 15.750000 V 13:11:24 - Starting flashes... 13:11:30 - Stopping flashes... 13:11:41 - Setting voltage to 16.000000 V 13:11:42 - Starting flashes... 13:11:48 - Stopping flashes... 13:11:49 - Finished task "Scan voltages" 13:15:18 - Stopping TIB... 13:15:18 - Status: TibState.Running 13:15:18 - Alarms: AlarmVector.ReadError 13:15:18 - Connection status: True TXclient_alive: True 13:15:18 - Disabling trigger 13:15:18 - Disabling PPS 13:15:19 - Status: TibState.PpsStopped 13:15:19 - Alarms: AlarmVector.ReadError 13:15:19 - Connection status: True TXclient_alive: True 13:15:20 - Closing TIB connection to camera server 13:15:24 - Status: TibState.PpsStopped 13:15:24 - Alarms: AlarmVector.ReadError 13:15:24 - Connection status: False TXclient_alive: False 13:15:24 - Run duration: 0:13:51 13:15:24 - Local camera triggers: 0 13:15:24 - Slow control triggers: 0 13:15:24 - Single photo-electron triggers: 0 13:15:24 - Calibration triggers: 206183 13:15:24 - Software triggers from UCTS: 0 13:15:24 - Pedestal triggers: 0 13:15:24 - Collected input triggers: 206183 13:15:24 - Busy triggers: 0 13:15:24 - Camera event triggers sent to clusters: 206183 13:15:27 - Calling method FSM.gotoReady in object FSM 13:15:29 - Event builder is in state CameraServerState.Observing and has output 206183 event(s) during run #2684. 13:15:29 - Event builder is in transition to state CameraServerState.Ready 13:15:33 - Stopping NMC 13:15:33 - ("NmcError: Failed to call 'Stop': status Bad statuses are present, code 0x80020000", 2147614720) 13:15:36 - NMC is in state NmcState.Ready and knows of 0 drawer(s)