15:03 - Disabling high voltages...
15:03 - Starting task "Disable HV"
15:03 - Maximum reference voltage is now 0.00 V
15:03 - Finished task "Disable HV"
15:03 - Setting HVPA ramp sped to 500 V/s
15:03 - Switching HVPA to state Wait
15:03 - Switching drawer 127 HVPA to state Wait
15:03 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
15:03 - Switching drawer 128 HVPA to state Wait
15:03 - Checking high voltages...
15:03 - Starting task "Check HV"
15:03 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:03 - Reference voltage of channel 0 in drawer #127: expected 1177.44, read 0.00
15:03 - Reference voltage of channel 1 in drawer #127: expected 1089.55, read 0.00
15:03 - Reference voltage of channel 2 in drawer #127: expected 1128.57, read 0.00
15:03 - Reference voltage of channel 3 in drawer #127: expected 1202.69, read 0.00
15:03 - Reference voltage of channel 4 in drawer #127: expected 1224.81, read 0.00
15:03 - Reference voltage of channel 5 in drawer #127: expected 1179.79, read 0.00
15:03 - Reference voltage of channel 6 in drawer #127: expected 1360.86, read 0.00
15:03 - Reference voltage of channel 0 in drawer #128: expected 966.45, read 0.00
15:03 - Reference voltage of channel 1 in drawer #128: expected 1001.83, read 0.00
15:03 - Reference voltage of channel 2 in drawer #128: expected 998.96, read 0.00
15:03 - Reference voltage of channel 3 in drawer #128: expected 1000.64, read 0.00
15:03 - Reference voltage of channel 4 in drawer #128: expected 1009.92, read 0.00
15:03 - Reference voltage of channel 5 in drawer #128: expected 991.82, read 0.00
15:03 - Reference voltage of channel 6 in drawer #128: expected 984.57, read 0.00
15:03 - Done
15:03 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:03 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 741.15 V and 1030.90 V.
15:03 - Maximum difference between expected and actual voltages in drawer 127 is 821.75 V
15:03 - Maximum difference between expected and actual voltages in drawer 128 is 767.38 V
15:03 - Checking status of HVPA module in 2 drawer(s)...
15:03 - Checking status of HVPA channels in 2 drawer(s)...
15:03 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 14 channel(s)
15:03 - Finished task "Check HV"
15:03 - Checking high voltages...
15:03 - Starting task "Check HV"
15:03 - Checking reference voltages against 0.000000 V
15:03 - Low voltage of drawer #127: expected 100.00, read 200.00
15:03 - Low voltage of drawer #128: expected 100.00, read 200.00
15:03 - Done
15:03 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:03 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 535.58 V and 681.98 V.
15:03 - Maximum difference between expected and actual voltages in drawer 127 is 477.71 V
15:03 - Maximum difference between expected and actual voltages in drawer 128 is 547.78 V
15:03 - Checking status of HVPA module in 2 drawer(s)...
15:03 - Checking status of HVPA channels in 2 drawer(s)...
15:03 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 14 channel(s)
15:03 - Finished task "Check HV"
15:03 - Checking high voltages...
15:03 - Starting task "Check HV"
15:03 - Checking reference voltages against 0.000000 V
15:03 - Low voltage of drawer #127: expected 100.00, read 200.00
15:03 - Low voltage of drawer #128: expected 100.00, read 200.00
15:03 - Done
15:03 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:03 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 505.08 V and 636.23 V.
15:03 - Maximum difference between expected and actual voltages in drawer 127 is 431.96 V
15:03 - Maximum difference between expected and actual voltages in drawer 128 is 516.06 V
15:03 - Checking status of HVPA module in 2 drawer(s)...
15:03 - Checking status of HVPA channels in 2 drawer(s)...
15:03 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 14 channel(s)
15:03 - Finished task "Check HV"
15:03 - Checking high voltages...
15:03 - Starting task "Check HV"
15:03 - Checking reference voltages against 0.000000 V
15:03 - Low voltage of drawer #127: expected 100.00, read 200.00
15:03 - Low voltage of drawer #128: expected 100.00, read 200.00
15:03 - Done
15:03 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:03 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 455.67 V and 561.81 V.
15:03 - Maximum difference between expected and actual voltages in drawer 127 is 358.15 V
15:03 - Maximum difference between expected and actual voltages in drawer 128 is 464.21 V
15:03 - Checking status of HVPA module in 2 drawer(s)...
15:03 - Checking status of HVPA channels in 2 drawer(s)...
15:03 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 9 channel(s)
15:03 - Finished task "Check HV"
15:03 - Checking high voltages...
15:03 - Starting task "Check HV"
15:03 - Checking reference voltages against 0.000000 V
15:03 - Low voltage of drawer #127: expected 100.00, read 200.00
15:03 - Low voltage of drawer #128: expected 100.00, read 200.00
15:03 - Done
15:03 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:04 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 431.88 V and 527.04 V.
15:04 - Maximum difference between expected and actual voltages in drawer 127 is 324.60 V
15:04 - Maximum difference between expected and actual voltages in drawer 128 is 438.59 V
15:04 - Checking status of HVPA module in 2 drawer(s)...
15:04 - Checking status of HVPA channels in 2 drawer(s)...
15:04 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 8 channel(s)
15:04 - Finished task "Check HV"
15:04 - Checking high voltages...
15:04 - Starting task "Check HV"
15:04 - Checking reference voltages against 0.000000 V
15:04 - Low voltage of drawer #127: expected 100.00, read 200.00
15:04 - Low voltage of drawer #128: expected 100.00, read 200.00
15:04 - Done
15:04 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:04 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 408.70 V and 494.10 V.
15:04 - Maximum difference between expected and actual voltages in drawer 127 is 290.44 V
15:04 - Maximum difference between expected and actual voltages in drawer 128 is 413.58 V
15:04 - Checking status of HVPA module in 2 drawer(s)...
15:04 - Checking status of HVPA channels in 2 drawer(s)...
15:04 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:04 - Finished task "Check HV"
15:04 - Checking high voltages...
15:04 - Starting task "Check HV"
15:04 - Checking reference voltages against 0.000000 V
15:04 - Low voltage of drawer #127: expected 100.00, read 200.00
15:04 - Low voltage of drawer #128: expected 100.00, read 200.00
15:04 - Done
15:04 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:04 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 380.64 V and 455.67 V.
15:04 - Maximum difference between expected and actual voltages in drawer 127 is 253.23 V
15:04 - Maximum difference between expected and actual voltages in drawer 128 is 385.52 V
15:04 - Checking status of HVPA module in 2 drawer(s)...
15:04 - Checking status of HVPA channels in 2 drawer(s)...
15:04 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:04 - Finished task "Check HV"
15:04 - Checking high voltages...
15:04 - Starting task "Check HV"
15:04 - Checking reference voltages against 0.000000 V
15:04 - Low voltage of drawer #127: expected 100.00, read 200.00
15:04 - Low voltage of drawer #128: expected 100.00, read 200.00
15:04 - Done
15:04 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:04 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 358.07 V and 423.95 V.
15:04 - Maximum difference between expected and actual voltages in drawer 127 is 222.12 V
15:04 - Maximum difference between expected and actual voltages in drawer 128 is 362.34 V
15:04 - Checking status of HVPA module in 2 drawer(s)...
15:04 - Checking status of HVPA channels in 2 drawer(s)...
15:04 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:04 - Finished task "Check HV"
15:04 - Checking high voltages...
15:04 - Starting task "Check HV"
15:04 - Checking reference voltages against 0.000000 V
15:04 - Low voltage of drawer #127: expected 100.00, read 200.00
15:04 - Low voltage of drawer #128: expected 100.00, read 200.00
15:04 - Done
15:04 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:04 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 340.99 V and 400.77 V.
15:04 - Maximum difference between expected and actual voltages in drawer 127 is 199.55 V
15:04 - Maximum difference between expected and actual voltages in drawer 128 is 344.04 V
15:04 - Checking status of HVPA module in 2 drawer(s)...
15:04 - Checking status of HVPA channels in 2 drawer(s)...
15:04 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:04 - Finished task "Check HV"
15:04 - Checking high voltages...
15:04 - Starting task "Check HV"
15:04 - Checking reference voltages against 0.000000 V
15:04 - Low voltage of drawer #127: expected 100.00, read 200.00
15:04 - Low voltage of drawer #128: expected 100.00, read 200.00
15:04 - Done
15:04 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:04 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 314.76 V and 367.22 V.
15:04 - Maximum difference between expected and actual voltages in drawer 127 is 165.39 V
15:04 - Maximum difference between expected and actual voltages in drawer 128 is 318.42 V
15:04 - Checking status of HVPA module in 2 drawer(s)...
15:04 - Checking status of HVPA channels in 2 drawer(s)...
15:04 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:04 - Finished task "Check HV"
15:04 - Checking high voltages...
15:04 - Starting task "Check HV"
15:04 - Checking reference voltages against 0.000000 V
15:04 - Low voltage of drawer #127: expected 100.00, read 200.00
15:04 - Low voltage of drawer #128: expected 100.00, read 200.00
15:04 - Done
15:04 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:04 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 301.95 V and 350.14 V.
15:04 - Maximum difference between expected and actual voltages in drawer 127 is 148.31 V
15:04 - Maximum difference between expected and actual voltages in drawer 128 is 304.39 V
15:04 - Checking status of HVPA module in 2 drawer(s)...
15:04 - Checking status of HVPA channels in 2 drawer(s)...
15:04 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:04 - Finished task "Check HV"
15:04 - Checking high voltages...
15:04 - Starting task "Check HV"
15:04 - Checking reference voltages against 0.000000 V
15:04 - Low voltage of drawer #127: expected 100.00, read 200.00
15:04 - Low voltage of drawer #128: expected 100.00, read 200.00
15:04 - Done
15:04 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:04 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 287.31 V and 331.84 V.
15:04 - Maximum difference between expected and actual voltages in drawer 127 is 130.62 V
15:04 - Maximum difference between expected and actual voltages in drawer 128 is 290.36 V
15:04 - Checking status of HVPA module in 2 drawer(s)...
15:04 - Checking status of HVPA channels in 2 drawer(s)...
15:04 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:04 - Finished task "Check HV"
15:04 - Checking high voltages...
15:04 - Starting task "Check HV"
15:04 - Checking reference voltages against 0.000000 V
15:04 - Low voltage of drawer #127: expected 100.00, read 200.00
15:04 - Low voltage of drawer #128: expected 100.00, read 200.00
15:04 - Done
15:04 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:04 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 273.89 V and 314.76 V.
15:04 - Maximum difference between expected and actual voltages in drawer 127 is 113.54 V
15:04 - Maximum difference between expected and actual voltages in drawer 128 is 276.94 V
15:04 - Checking status of HVPA module in 2 drawer(s)...
15:04 - Checking status of HVPA channels in 2 drawer(s)...
15:04 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:04 - Finished task "Check HV"
15:31 - Checking high voltages...
15:31 - Starting task "Check HV"
15:31 - Checking reference voltages against 0.000000 V
15:31 - Low voltage of drawer #127: expected 100.00, read 200.00
15:31 - Low voltage of drawer #128: expected 100.00, read 200.00
15:31 - Done
15:31 - Found 0 drawer(s) in Nominal state, 1 in Low state and 1 in Wait state
15:31 - Found 2 drawer(s). Reference voltage is between 0.00 V and 0.00 V. Actual voltage is between 0.00 V and 3.66 V.
15:31 - Maximum difference between expected and actual voltages in drawer 127 is 200.00 V
15:31 - Maximum difference between expected and actual voltages in drawer 128 is 3.66 V
15:31 - Checking status of HVPA module in 2 drawer(s)...
15:31 - Checking status of HVPA channels in 2 drawer(s)...
15:31 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:31 - Finished task "Check HV"
15:31 - Starting task "Configure NMC"
15:31 - Configuring modules with configuration /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:31 - NMC is in state NmcState.Configured and knows of 2 drawer(s)
15:31 - Found 0 PPS errors.
15:31 - Finished task "Configure NMC"
15:32 - Setting HVPA ramp sped to 20 V/s
15:32 - Switching HVPA to state Nominal
15:32 - Switching drawer 127 HVPA to state Nominal
15:32 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
15:32 - Switching drawer 128 HVPA to state Nominal
15:32 - Setting HVPA ramp sped to 20 V/s
15:32 - Switching HVPA to state Nominal
15:32 - Switching drawer 127 HVPA to state Nominal
15:32 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
15:32 - Switching drawer 128 HVPA to state Nominal
15:32 - Checking high voltages...
15:32 - Starting task "Check HV"
15:32 - Checking reference voltages against 0.000000 V
15:32 - Reference voltage of channel 0 in drawer #127: expected 0.00, read 1177.44
15:32 - Reference voltage of channel 1 in drawer #127: expected 0.00, read 1089.55
15:32 - Reference voltage of channel 2 in drawer #127: expected 0.00, read 1128.57
15:32 - Reference voltage of channel 3 in drawer #127: expected 0.00, read 1202.69
15:32 - Reference voltage of channel 4 in drawer #127: expected 0.00, read 1224.81
15:32 - Reference voltage of channel 5 in drawer #127: expected 0.00, read 1179.79
15:32 - Reference voltage of channel 6 in drawer #127: expected 0.00, read 1360.86
15:32 - Low voltage of drawer #127: expected 100.00, read 200.00
15:32 - Reference voltage of channel 0 in drawer #128: expected 0.00, read 966.45
15:32 - Reference voltage of channel 1 in drawer #128: expected 0.00, read 1001.83
15:32 - Reference voltage of channel 2 in drawer #128: expected 0.00, read 998.96
15:32 - Reference voltage of channel 3 in drawer #128: expected 0.00, read 1000.64
15:32 - Reference voltage of channel 4 in drawer #128: expected 0.00, read 1009.92
15:32 - Reference voltage of channel 5 in drawer #128: expected 0.00, read 991.82
15:32 - Reference voltage of channel 6 in drawer #128: expected 0.00, read 984.57
15:32 - Low voltage of drawer #128: expected 100.00, read 200.00
15:32 - Done
15:32 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:32 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 0.61 V and 782.02 V.
15:32 - Maximum difference between expected and actual voltages in drawer 127 is 585.68 V
15:32 - Maximum difference between expected and actual voltages in drawer 128 is 983.96 V
15:32 - Checking status of HVPA module in 2 drawer(s)...
15:32 - HVPA module in drawer 128 has a status of 0x3f!
15:32 - Checking status of HVPA channels in 2 drawer(s)...
15:32 - HVPA channel 6 of drawer 128 has status 0x8003!
15:32 - Resetting HVPA channel 6 of drawer 128...
15:32 - HV was locked in 1 channel(s), was in rise default in 0 channel(s) and was still in progress in 13 channel(s)
15:32 - Finished task "Check HV"
15:32 - Checking high voltages...
15:32 - Starting task "Check HV"
15:32 - Checking reference voltages against 0.000000 V
15:32 - Reference voltage of channel 0 in drawer #127: expected 0.00, read 1177.44
15:32 - Reference voltage of channel 1 in drawer #127: expected 0.00, read 1089.55
15:32 - Reference voltage of channel 2 in drawer #127: expected 0.00, read 1128.57
15:32 - Reference voltage of channel 3 in drawer #127: expected 0.00, read 1202.69
15:32 - Reference voltage of channel 4 in drawer #127: expected 0.00, read 1224.81
15:32 - Reference voltage of channel 5 in drawer #127: expected 0.00, read 1179.79
15:32 - Reference voltage of channel 6 in drawer #127: expected 0.00, read 1360.86
15:32 - Low voltage of drawer #127: expected 100.00, read 200.00
15:32 - Reference voltage of channel 0 in drawer #128: expected 0.00, read 966.45
15:32 - Reference voltage of channel 1 in drawer #128: expected 0.00, read 1001.83
15:32 - Reference voltage of channel 2 in drawer #128: expected 0.00, read 998.96
15:32 - Reference voltage of channel 3 in drawer #128: expected 0.00, read 1000.64
15:32 - Reference voltage of channel 4 in drawer #128: expected 0.00, read 1009.92
15:32 - Reference voltage of channel 5 in drawer #128: expected 0.00, read 991.82
15:32 - Low voltage of drawer #128: expected 100.00, read 200.00
15:32 - Done
15:32 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:32 - Found 2 drawer(s). Reference voltage is between 0.00 V and 1360.86 V. Actual voltage is between 1.83 V and 1040.66 V.
15:32 - Maximum difference between expected and actual voltages in drawer 127 is 844.32 V
15:32 - Maximum difference between expected and actual voltages in drawer 128 is 91.26 V
15:32 - Checking status of HVPA module in 2 drawer(s)...
15:32 - HVPA module in drawer 128 has a status of 0x3f!
15:32 - Checking status of HVPA channels in 2 drawer(s)...
15:32 - HVPA channel 6 of drawer 128 has status 0x8003!
15:32 - Resetting HVPA channel 6 of drawer 128...
15:32 - HV was locked in 1 channel(s), was in rise default in 0 channel(s) and was still in progress in 13 channel(s)
15:32 - Finished task "Check HV"
15:32 - Setting HVPA ramp sped to 500 V/s
15:32 - Switching HVPA to state Low
15:32 - Switching drawer 127 HVPA to state Low
15:32 - Switching drawer 128 HVPA to state Low
15:33 - Stopping NMC
15:33 - Found 0 PPS errors.
15:33 - Resetting NMC
15:33 - Starting NMC
15:33 - Clearing PPS error counters
15:33 - Failed to read 'Nf': No targets are given
15:33 - Starting task "Configure NMC"
15:33 - Configuring modules with configuration /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:34 - NMC is in state NmcState.Configured and knows of 2 drawer(s)
15:34 - Found 0 PPS errors.
15:34 - Finished task "Configure NMC"
15:34 - Setting HVPA ramp sped to 20 V/s
15:34 - Switching HVPA to state Nominal
15:34 - Switching drawer 127 HVPA to state Nominal
15:34 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
15:34 - Switching drawer 128 HVPA to state Nominal
15:34 - Checking high voltages...
15:34 - Starting task "Check HV"
15:34 - Checking reference voltages against 0.000000 V
15:34 - Reference voltage of channel 0 in drawer #127: expected 0.00, read 1177.44
15:34 - Reference voltage of channel 1 in drawer #127: expected 0.00, read 1089.55
15:34 - Reference voltage of channel 2 in drawer #127: expected 0.00, read 1128.57
15:34 - Reference voltage of channel 3 in drawer #127: expected 0.00, read 1202.69
15:34 - Reference voltage of channel 4 in drawer #127: expected 0.00, read 1224.81
15:34 - Reference voltage of channel 5 in drawer #127: expected 0.00, read 1179.79
15:34 - Reference voltage of channel 6 in drawer #127: expected 0.00, read 1360.86
15:34 - Low voltage of drawer #127: expected 100.00, read 200.00
15:34 - Reference voltage of channel 0 in drawer #128: expected 0.00, read 966.45
15:34 - Reference voltage of channel 1 in drawer #128: expected 0.00, read 1001.83
15:34 - Reference voltage of channel 2 in drawer #128: expected 0.00, read 998.96
15:34 - Reference voltage of channel 3 in drawer #128: expected 0.00, read 1000.64
15:34 - Reference voltage of channel 4 in drawer #128: expected 0.00, read 1009.92
15:34 - Reference voltage of channel 5 in drawer #128: expected 0.00, read 991.82
15:34 - Reference voltage of channel 6 in drawer #128: expected 0.00, read 984.57
15:34 - Low voltage of drawer #128: expected 100.00, read 200.00
15:34 - Done
15:34 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:34 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 0.00 V and 1361.52 V.
15:34 - Maximum difference between expected and actual voltages in drawer 127 is 1161.52 V
15:34 - Maximum difference between expected and actual voltages in drawer 128 is 984.57 V
15:34 - Checking status of HVPA module in 2 drawer(s)...
15:34 - HVPA module in drawer 128 has a status of 0x3f!
15:34 - Checking status of HVPA channels in 2 drawer(s)...
15:34 - HVPA channel 6 of drawer 128 has status 0x8003!
15:34 - Resetting HVPA channel 6 of drawer 128...
15:34 - HV was locked in 1 channel(s), was in rise default in 0 channel(s) and was still in progress in 6 channel(s)
15:34 - Finished task "Check HV"
15:34 - NMC is in state NmcState.Configured and knows of 2 drawer(s)
15:34 - Found 0 PPS errors.
15:34 - Enabling high voltages...
15:34 - Starting task "Enable HV"
15:34 - Setting HV to 100.0 V
15:34 - Check of HVPA failed: expected 100.00, read 1177.44
15:34 - Check of HVPA failed: expected 100.00, read 1224.81
15:34 - Checking status of HVPA channels in 2 drawer(s)...
15:34 - HVPA channel 6 of drawer 128 has status 0x8003!
15:34 - Resetting HVPA channel 6 of drawer 128...
15:34 - HV was locked in 1 channel(s), was in rise default in 0 channel(s) and was still in progress in 13 channel(s)
15:34 - Setting HV to nominal values...
15:34 - Setting channel 0 voltages to [1177.44, 966.45]
15:34 - Setting channel 1 voltages to [1089.55, 1001.83]
15:34 - Setting channel 2 voltages to [1128.57, 998.96]
15:34 - Setting channel 3 voltages to [1202.69, 1000.64]
15:34 - Setting channel 4 voltages to [1224.81, 1009.92]
15:34 - Setting channel 5 voltages to [1179.79, 991.82]
15:34 - Setting channel 6 voltages to [1360.86, 984.57]
15:34 - Checking status of HVPA channels in 2 drawer(s)...
15:34 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 11 channel(s)
15:34 - Finished task "Enable HV"
15:34 - Checking high voltages...
15:34 - Starting task "Check HV"
15:34 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:34 - Done
15:34 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:34 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 246.44 V and 1360.91 V.
15:34 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:34 - Maximum difference between expected and actual voltages in drawer 128 is 728.98 V
15:34 - Checking status of HVPA module in 2 drawer(s)...
15:34 - Checking status of HVPA channels in 2 drawer(s)...
15:34 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:34 - Finished task "Check HV"
15:35 - Checking high voltages...
15:35 - Starting task "Check HV"
15:35 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:35 - Done
15:35 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:35 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 605.73 V and 1360.91 V.
15:35 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:35 - Maximum difference between expected and actual voltages in drawer 128 is 370.30 V
15:35 - Checking status of HVPA module in 2 drawer(s)...
15:35 - Checking status of HVPA channels in 2 drawer(s)...
15:35 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:35 - Finished task "Check HV"
15:35 - Checking high voltages...
15:35 - Starting task "Check HV"
15:35 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:35 - Done
15:35 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:35 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 904.63 V and 1360.91 V.
15:35 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:35 - Maximum difference between expected and actual voltages in drawer 128 is 71.40 V
15:35 - Checking status of HVPA module in 2 drawer(s)...
15:35 - Checking status of HVPA channels in 2 drawer(s)...
15:35 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:35 - Finished task "Check HV"
15:35 - Checking high voltages...
15:35 - Starting task "Check HV"
15:35 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:35 - Done
15:35 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:35 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:35 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:35 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:35 - Checking status of HVPA module in 2 drawer(s)...
15:35 - Checking status of HVPA channels in 2 drawer(s)...
15:35 - HV is stabilized in all channels.
15:35 - Finished task "Check HV"
15:35 - Checking high voltages...
15:35 - Starting task "Check HV"
15:35 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:35 - Done
15:35 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:35 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:35 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:35 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:35 - Checking status of HVPA module in 2 drawer(s)...
15:35 - Checking status of HVPA channels in 2 drawer(s)...
15:35 - HV is stabilized in all channels.
15:35 - Finished task "Check HV"
15:38 - Checking high voltages...
15:38 - Starting task "Check HV"
15:38 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:38 - Done
15:38 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:38 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:38 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:38 - Maximum difference between expected and actual voltages in drawer 128 is 0.65 V
15:38 - Checking status of HVPA module in 2 drawer(s)...
15:38 - Checking status of HVPA channels in 2 drawer(s)...
15:38 - HV is stabilized in all channels.
15:38 - Finished task "Check HV"
15:39 - Checking high voltages...
15:39 - Starting task "Check HV"
15:39 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:39 - Done
15:39 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:39 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:39 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:39 - Maximum difference between expected and actual voltages in drawer 128 is 0.40 V
15:39 - Checking status of HVPA module in 2 drawer(s)...
15:39 - Checking status of HVPA channels in 2 drawer(s)...
15:39 - HV is stabilized in all channels.
15:39 - Finished task "Check HV"
15:39 - Status: TibState.PpsStopped
15:39 - Alarms: AlarmVector.ReadError
15:39 - Connection status: False
15:39 - External delays: Local=0.000 ns, Calibration=76.000 ns, Sphe=0.000 ns
15:39 - Trigger mask: 0x44 = TriggerMask.SlowControl|Calibration
15:39 - Input mask: 0x0
15:39 - Disable BUSY: False
15:39 - Extra BUSY Delay: 0.0
15:39 - Temperature: 38.62
15:39 - Configuring TIB...
15:39 - Setting TIB forced mode to False
15:39 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
15:39 - Status: TibState.PpsStopped
15:39 - Alarms: AlarmVector.ReadError
15:39 - Connection status: False
15:39 - Entering set_external_delay
15:39 - Setting ExternalDelayLocal to 0.000 ns
15:39 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
15:39 - Entering set_external_delay
15:39 - Setting ExternalDelayCalibration to 76.000 ns
15:39 - Writing 76.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
15:39 - Entering set_external_delay
15:39 - Setting ExternalDelaySphe to 0.000 ns
15:39 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
15:39 - Setting TIB trigger mode to 0x44 = TriggerMask.SlowControl|Calibration
15:39 - Writing 68 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
15:39 - Writing 0 to Camera.TIB.InputsEnable.InputMask.InputMask_v
15:39 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
15:39 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
15:39 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
15:39 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
15:39 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
15:39 - Status: TibState.PpsStopped
15:39 - Alarms: AlarmVector.ReadError
15:39 - Connection status: False
15:39 - Writing False to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
15:39 - Writing 10000 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
15:39 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:39 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
15:39 - Writing 10000 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
15:39 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
15:39 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:39 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
15:39 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
15:39 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:39 - Connecting TIB to camera server at 10.10.4.12:222
15:39 - Starting TIB...
15:39 - Status: TibState.PpsStopped
15:39 - Alarms: AlarmVector.ReadError
15:39 - Connection status: False
15:39 - Resetting run
15:39 - Wait for UCTS external trigger before enabling trigger...
15:39 - Status: TibState.Standby
15:39 - Alarms: AlarmVector.NoAlarm
15:39 - Connection status: False
15:39 - Status: TibState.Standby
15:39 - Alarms: AlarmVector.NoAlarm
15:39 - Connection status: False
15:39 - Status: TibState.Armed
15:39 - Alarms: AlarmVector.NoAlarm
15:39 - Connection status: True
15:39 - Starting TIB triggers...
15:39 - Enabling trigger
15:39 - Status: TibState.Running
15:39 - Alarms: AlarmVector.NoAlarm
15:39 - Connection status: True
15:40 - Configuring calibration box...
15:40 - Configuring calibration box with leds=0x40 voltage=12.00 frequency=5000.0/1 central_current=0.0 duration=400 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=False enabled=False
15:40 - Starting flashes...
15:43 - Resetting NMC
15:43 - Starting task "Configure NMC"
15:43 - Configuring modules with configuration /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:43 - Checking high voltages...
15:43 - NMC is in state NmcState.Configured and knows of 2 drawer(s)
15:43 - Found 0 PPS errors.
15:43 - Finished task "Check HV"
15:43 - Checking high voltages...
15:43 - Starting task "Check HV"
15:43 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:43 - Done
15:43 - Found 0 drawer(s) in Nominal state, 2 in Low state and 0 in Wait state
15:43 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 838.75 V and 1360.91 V.
15:43 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:43 - Maximum difference between expected and actual voltages in drawer 128 is 666.81 V
15:43 - Checking status of HVPA module in 2 drawer(s)...
15:43 - Checking status of HVPA channels in 2 drawer(s)...
15:43 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:43 - Finished task "Check HV"
15:43 - Setting HVPA ramp sped to 20 V/s
15:43 - Switching HVPA to state Nominal
15:43 - Switching drawer 127 HVPA to state Nominal
15:43 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
15:43 - Switching drawer 128 HVPA to state Nominal
15:43 - Checking high voltages...
15:43 - Starting task "Check HV"
15:43 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:43 - Done
15:43 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:43 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 667.34 V and 1360.91 V.
15:43 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:43 - Maximum difference between expected and actual voltages in drawer 128 is 297.28 V
15:43 - Checking status of HVPA module in 2 drawer(s)...
15:43 - Checking status of HVPA channels in 2 drawer(s)...
15:43 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:43 - Finished task "Check HV"
15:44 - Checking high voltages...
15:44 - Starting task "Check HV"
15:44 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:44 - Done
15:44 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:44 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 965.63 V and 1360.91 V.
15:44 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:44 - Maximum difference between expected and actual voltages in drawer 128 is 0.57 V
15:44 - Checking status of HVPA module in 2 drawer(s)...
15:44 - Checking status of HVPA channels in 2 drawer(s)...
15:44 - HV is stabilized in all channels.
15:44 - Finished task "Check HV"
15:44 - Checking high voltages...
15:44 - Starting task "Check HV"
15:44 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:44 - Done
15:44 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:44 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:44 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:44 - Maximum difference between expected and actual voltages in drawer 128 is 0.57 V
15:44 - Checking status of HVPA module in 2 drawer(s)...
15:44 - Checking status of HVPA channels in 2 drawer(s)...
15:44 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:44 - Finished task "Check HV"
15:44 - Checking high voltages...
15:44 - Starting task "Check HV"
15:44 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:44 - Done
15:44 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:44 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:44 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:44 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:44 - Checking status of HVPA module in 2 drawer(s)...
15:44 - Checking status of HVPA channels in 2 drawer(s)...
15:44 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:44 - Finished task "Check HV"
15:44 - Checking high voltages...
15:44 - Starting task "Check HV"
15:44 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:44 - Done
15:44 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:44 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.30 V.
15:44 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:44 - Maximum difference between expected and actual voltages in drawer 128 is 0.40 V
15:44 - Checking status of HVPA module in 2 drawer(s)...
15:44 - Checking status of HVPA channels in 2 drawer(s)...
15:44 - HV is stabilized in all channels.
15:44 - Finished task "Check HV"
15:44 - Checking high voltages...
15:44 - Starting task "Check HV"
15:44 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:44 - Done
15:44 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:44 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:44 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:44 - Maximum difference between expected and actual voltages in drawer 128 is 0.40 V
15:44 - Checking status of HVPA module in 2 drawer(s)...
15:44 - Checking status of HVPA channels in 2 drawer(s)...
15:44 - HV is stabilized in all channels.
15:44 - Finished task "Check HV"
15:44 - Checking high voltages...
15:44 - Starting task "Check HV"
15:44 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:44 - Done
15:44 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:44 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.85 V and 1360.91 V.
15:44 - Maximum difference between expected and actual voltages in drawer 127 is 1161.52 V
15:44 - Maximum difference between expected and actual voltages in drawer 128 is 0.40 V
15:44 - Checking status of HVPA module in 2 drawer(s)...
15:44 - Checking status of HVPA channels in 2 drawer(s)...
15:44 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:44 - Finished task "Check HV"
15:45 - Checking high voltages...
15:45 - Starting task "Check HV"
15:45 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:45 - Done
15:45 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:45 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.85 V and 1360.91 V.
15:45 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:45 - Maximum difference between expected and actual voltages in drawer 128 is 0.39 V
15:45 - Checking status of HVPA module in 2 drawer(s)...
15:45 - Checking status of HVPA channels in 2 drawer(s)...
15:45 - HV is stabilized in all channels.
15:45 - Finished task "Check HV"
15:45 - Checking high voltages...
15:45 - Starting task "Check HV"
15:45 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:45 - Done
15:45 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:45 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:45 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:45 - Maximum difference between expected and actual voltages in drawer 128 is 0.39 V
15:45 - Checking status of HVPA module in 2 drawer(s)...
15:45 - Checking status of HVPA channels in 2 drawer(s)...
15:45 - HV is stabilized in all channels.
15:45 - Finished task "Check HV"
15:45 - Checking high voltages...
15:45 - Starting task "Check HV"
15:45 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:45 - Done
15:45 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:45 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:45 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:45 - Maximum difference between expected and actual voltages in drawer 128 is 0.40 V
15:45 - Checking status of HVPA module in 2 drawer(s)...
15:45 - Checking status of HVPA channels in 2 drawer(s)...
15:45 - HV is stabilized in all channels.
15:45 - Finished task "Check HV"
15:46 - Checking high voltages...
15:46 - Starting task "Check HV"
15:46 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:46 - Done
15:46 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:46 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:46 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:46 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:46 - Checking status of HVPA module in 2 drawer(s)...
15:46 - Checking status of HVPA channels in 2 drawer(s)...
15:46 - HV is stabilized in all channels.
15:46 - Finished task "Check HV"
15:46 - Starting NMC
15:46 - Clearing PPS error counters
15:46 - Configuring TIB...
15:46 - Setting TIB forced mode to False
15:46 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
15:46 - Status: TibState.Running
15:46 - Alarms: AlarmVector.ReadError
15:46 - Connection status: True
15:46 - Disabling PPS
15:46 - Error configuring TIB: Failed to call DisablePPS: Bad statuses are present
15:47 - Configuring TIB...
15:47 - Setting TIB forced mode to False
15:47 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
15:47 - Status: TibState.Running
15:47 - Alarms: AlarmVector.ReadError
15:47 - Connection status: True
15:47 - Disabling PPS
15:47 - Error configuring TIB: Failed to call DisablePPS: Bad statuses are present
15:47 - Resetting TIB...
15:47 - Failed to call Reset: Bad statuses are present
15:47 - Configuring TIB...
15:47 - Setting TIB forced mode to False
15:47 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
15:47 - Status: TibState.RunStopped
15:47 - Alarms: AlarmVector.NoAlarm
15:47 - Connection status: False
15:47 - Disabling PPS
15:47 - Status: TibState.PpsStopped
15:47 - Alarms: AlarmVector.NoAlarm
15:47 - Connection status: False
15:47 - Entering set_external_delay
15:47 - Setting ExternalDelayLocal to 0.000 ns
15:47 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
15:47 - Entering set_external_delay
15:47 - Setting ExternalDelayCalibration to 76.000 ns
15:47 - Writing 76.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
15:47 - Entering set_external_delay
15:47 - Setting ExternalDelaySphe to 0.000 ns
15:47 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
15:47 - Setting TIB trigger mode to 0x44 = TriggerMask.SlowControl|Calibration
15:47 - Writing 68 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
15:47 - Writing 0 to Camera.TIB.InputsEnable.InputMask.InputMask_v
15:47 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
15:47 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
15:47 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
15:47 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
15:47 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
15:47 - Status: TibState.PpsStopped
15:47 - Alarms: AlarmVector.NoAlarm
15:47 - Connection status: False
15:47 - Writing False to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
15:47 - Writing 10000 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
15:47 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:47 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
15:47 - Writing 10000 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
15:47 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
15:47 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:47 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
15:47 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
15:47 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
15:48 - Connecting TIB to camera server at 10.10.4.12:222
15:48 - Starting TIB...
15:48 - Status: TibState.PpsStopped
15:48 - Alarms: AlarmVector.NoAlarm
15:48 - Connection status: False
15:48 - Resetting run
15:48 - Wait for UCTS external trigger before enabling trigger...
15:48 - Status: TibState.Standby
15:48 - Alarms: AlarmVector.NoAlarm
15:48 - Connection status: False
15:48 - Status: TibState.Standby
15:48 - Alarms: AlarmVector.NoAlarm
15:48 - Connection status: False
15:48 - Status: TibState.Armed
15:48 - Alarms: AlarmVector.NoAlarm
15:48 - Connection status: True
15:48 - Starting TIB triggers...
15:48 - Enabling trigger
15:48 - Status: TibState.Running
15:48 - Alarms: AlarmVector.NoAlarm
15:48 - Connection status: True
15:48 - Configuring calibration box...
15:48 - Configuring calibration box with leds=0x40 voltage=12.00 frequency=5000.0/1 central_current=0.0 duration=400 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=False enabled=False
15:48 - Starting flashes...
15:52 - Resetting NMC
15:52 - Starting task "Configure NMC"
15:52 - Configuring modules with configuration /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:52 - Setting HVPA ramp sped to 20 V/s
15:52 - Switching HVPA to state Nominal
15:52 - NMC is in state NmcState.Configured and knows of 2 drawer(s)
15:52 - Found 0 PPS errors.
15:52 - Finished task "Configure NMC"
15:52 - Checking high voltages...
15:52 - Starting task "Check HV"
15:52 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:52 - Done
15:52 - Found 0 drawer(s) in Nominal state, 2 in Low state and 0 in Wait state
15:52 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 821.06 V and 1360.91 V.
15:52 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:52 - Maximum difference between expected and actual voltages in drawer 128 is 649.73 V
15:52 - Checking status of HVPA module in 2 drawer(s)...
15:52 - Checking status of HVPA channels in 2 drawer(s)...
15:52 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:52 - Finished task "Check HV"
15:52 - Checking high voltages...
15:52 - Starting task "Check HV"
15:52 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:52 - Done
15:52 - Found 0 drawer(s) in Nominal state, 2 in Low state and 0 in Wait state
15:52 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 710.04 V and 1360.91 V.
15:52 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:52 - Maximum difference between expected and actual voltages in drawer 128 is 539.32 V
15:52 - Checking status of HVPA module in 2 drawer(s)...
15:52 - Checking status of HVPA channels in 2 drawer(s)...
15:52 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:52 - Finished task "Check HV"
15:52 - Checking high voltages...
15:52 - Starting task "Check HV"
15:52 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:52 - Done
15:52 - Found 0 drawer(s) in Nominal state, 2 in Low state and 0 in Wait state
15:52 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 586.21 V and 1360.91 V.
15:52 - Maximum difference between expected and actual voltages in drawer 127 is 1161.52 V
15:52 - Maximum difference between expected and actual voltages in drawer 128 is 414.27 V
15:52 - Checking status of HVPA module in 2 drawer(s)...
15:52 - Checking status of HVPA channels in 2 drawer(s)...
15:52 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 7 channel(s)
15:52 - Finished task "Check HV"
15:52 - Setting HVPA ramp sped to 20 V/s
15:52 - Switching HVPA to state Nominal
15:52 - Switching drawer 127 HVPA to state Nominal
15:52 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
15:52 - Switching drawer 128 HVPA to state Nominal
15:53 - Checking high voltages...
15:53 - Starting task "Check HV"
15:53 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:53 - Reference voltage of channel 0 in drawer #127: expected 1177.44, read 0.00
15:53 - Done
15:53 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:53 - Found 2 drawer(s). Reference voltage is between 0.00 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:53 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:53 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:53 - Checking status of HVPA module in 2 drawer(s)...
15:53 - Checking status of HVPA channels in 2 drawer(s)...
15:53 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:53 - Finished task "Check HV"
15:53 - Checking high voltages...
15:53 - Starting task "Check HV"
15:53 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:53 - Reference voltage of channel 0 in drawer #127: expected 1177.44, read 0.00
15:53 - Done
15:53 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:53 - Found 2 drawer(s). Reference voltage is between 0.00 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:53 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:53 - Maximum difference between expected and actual voltages in drawer 128 is 0.39 V
15:53 - Checking status of HVPA module in 2 drawer(s)...
15:53 - Checking status of HVPA channels in 2 drawer(s)...
15:53 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:53 - Finished task "Check HV"
15:53 - Checking high voltages...
15:53 - Starting task "Check HV"
15:53 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:53 - Reference voltage of channel 0 in drawer #127: expected 1177.44, read 0.00
15:53 - Done
15:53 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:53 - Found 2 drawer(s). Reference voltage is between 0.00 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:53 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:53 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:53 - Checking status of HVPA module in 2 drawer(s)...
15:53 - Checking status of HVPA channels in 2 drawer(s)...
15:53 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:53 - Finished task "Check HV"
15:53 - Checking high voltages...
15:53 - Starting task "Check HV"
15:53 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:53 - Reference voltage of channel 0 in drawer #127: expected 1177.44, read 0.00
15:53 - Done
15:53 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:53 - Found 2 drawer(s). Reference voltage is between 0.00 V and 1360.86 V. Actual voltage is between 966.85 V and 1360.91 V.
15:53 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:53 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:53 - Checking status of HVPA module in 2 drawer(s)...
15:53 - Checking status of HVPA channels in 2 drawer(s)...
15:53 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:53 - Finished task "Check HV"
15:53 - Checking high voltages...
15:53 - Starting task "Check HV"
15:53 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:53 - Reference voltage of channel 0 in drawer #127: expected 1177.44, read 0.00
15:53 - Done
15:53 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:53 - Found 2 drawer(s). Reference voltage is between 0.00 V and 1360.86 V. Actual voltage is between 966.85 V and 1360.91 V.
15:53 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:53 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:53 - Checking status of HVPA module in 2 drawer(s)...
15:53 - Checking status of HVPA channels in 2 drawer(s)...
15:53 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:53 - Finished task "Check HV"
15:54 - Setting HVPA ramp sped to 20 V/s
15:54 - Switching HVPA to state Nominal
15:54 - Switching drawer 127 HVPA to state Nominal
15:54 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
15:54 - Switching drawer 128 HVPA to state Nominal
15:54 - Checking high voltages...
15:54 - Starting task "Check HV"
15:54 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:54 - Reference voltage of channel 0 in drawer #127: expected 1177.44, read 0.00
15:54 - Done
15:54 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:54 - Found 2 drawer(s). Reference voltage is between 0.00 V and 1360.86 V. Actual voltage is between 957.70 V and 1360.91 V.
15:54 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:54 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:54 - Checking status of HVPA module in 2 drawer(s)...
15:54 - Checking status of HVPA channels in 2 drawer(s)...
15:54 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 1 channel(s)
15:54 - Finished task "Check HV"
15:54 - Setting HVPA ramp sped to 500 V/s
15:54 - Switching HVPA to state Low
15:54 - Switching drawer 127 HVPA to state Low
15:54 - Switching drawer 128 HVPA to state Low
15:54 - Setting HVPA ramp sped to 20 V/s
15:54 - Switching HVPA to state Nominal
15:54 - Switching drawer 127 HVPA to state Nominal
15:54 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
15:54 - Switching drawer 128 HVPA to state Nominal
15:54 - Checking high voltages...
15:54 - Starting task "Check HV"
15:54 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:54 - Reference voltage of channel 0 in drawer #127: expected 1177.44, read 0.00
15:54 - Done
15:54 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:54 - Found 2 drawer(s). Reference voltage is between 0.00 V and 1360.86 V. Actual voltage is between 462.99 V and 1360.30 V.
15:54 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:54 - Maximum difference between expected and actual voltages in drawer 128 is 537.17 V
15:54 - Checking status of HVPA module in 2 drawer(s)...
15:54 - Checking status of HVPA channels in 2 drawer(s)...
15:54 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 8 channel(s)
15:54 - Finished task "Check HV"
15:54 - Stopping NMC
15:54 - Found 0 PPS errors.
15:54 - Resetting NMC
15:54 - Starting task "Configure NMC"
15:54 - Configuring modules with configuration /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:54 - NMC is in state NmcState.Configured and knows of 2 drawer(s)
15:54 - Found 0 PPS errors.
15:54 - Finished task "Configure NMC"
15:55 - Setting HVPA ramp sped to 20 V/s
15:55 - Switching HVPA to state Nominal
15:55 - Switching drawer 127 HVPA to state Nominal
15:55 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
15:55 - Switching drawer 128 HVPA to state Nominal
15:55 - Checking high voltages...
15:55 - Starting task "Check HV"
15:55 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:55 - Done
15:55 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:55 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 594.75 V and 1360.91 V.
15:55 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:55 - Maximum difference between expected and actual voltages in drawer 128 is 404.80 V
15:55 - Checking status of HVPA module in 2 drawer(s)...
15:55 - Checking status of HVPA channels in 2 drawer(s)...
15:55 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 8 channel(s)
15:55 - Finished task "Check HV"
15:56 - Checking high voltages...
15:56 - Starting task "Check HV"
15:56 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:56 - Done
15:56 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:56 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:56 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:56 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:56 - Checking status of HVPA module in 2 drawer(s)...
15:56 - Checking status of HVPA channels in 2 drawer(s)...
15:56 - HV is stabilized in all channels.
15:56 - Finished task "Check HV"
15:56 - Checking high voltages...
15:56 - Starting task "Check HV"
15:56 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
15:56 - Done
15:56 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
15:56 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
15:56 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
15:56 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
15:56 - Checking status of HVPA module in 2 drawer(s)...
15:56 - Checking status of HVPA channels in 2 drawer(s)...
15:56 - HV is stabilized in all channels.
15:56 - Finished task "Check HV"
15:56 - Starting NMC
15:56 - Clearing PPS error counters
16:00 - Stopping TIB...
16:00 - Status: TibState.Running
16:00 - Alarms: AlarmVector.ReadError
16:00 - Connection status: True
16:00 - Disabling trigger
16:00 - Disabling PPS
16:00 - Status: TibState.PpsStopped
16:00 - Alarms: AlarmVector.ReadError
16:00 - Connection status: True
16:00 - Closing TIB connection to camera server
16:04 - NMC is in state NmcState.Running and knows of 2 drawer(s)
16:04 - Found 0 PPS errors.
16:04 - Configuring TIB...
16:04 - Setting TIB forced mode to False
16:04 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
16:04 - Status: TibState.PpsStopped
16:04 - Alarms: AlarmVector.ReadError
16:04 - Connection status: False
16:04 - Entering set_external_delay
16:04 - Setting ExternalDelayLocal to 0.000 ns
16:04 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
16:04 - Entering set_external_delay
16:04 - Setting ExternalDelayCalibration to 76.000 ns
16:04 - Writing 76.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
16:04 - Entering set_external_delay
16:04 - Setting ExternalDelaySphe to 0.000 ns
16:04 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
16:04 - Setting TIB trigger mode to 0x44 = TriggerMask.SlowControl|Calibration
16:04 - Writing 68 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
16:04 - Writing 0 to Camera.TIB.InputsEnable.InputMask.InputMask_v
16:04 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
16:04 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
16:04 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
16:04 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
16:04 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
16:04 - Status: TibState.PpsStopped
16:04 - Alarms: AlarmVector.ReadError
16:04 - Connection status: False
16:04 - Writing False to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
16:04 - Writing 10000 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
16:04 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:04 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
16:04 - Writing 10000 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
16:04 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
16:04 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:04 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
16:04 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
16:04 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:04 - Connecting TIB to camera server at 10.10.4.12:222
16:04 - Starting TIB...
16:04 - Status: TibState.PpsStopped
16:04 - Alarms: AlarmVector.ReadError
16:04 - Connection status: False
16:04 - Resetting run
16:04 - Wait for UCTS external trigger before enabling trigger...
16:04 - Status: TibState.Standby
16:04 - Alarms: AlarmVector.NoAlarm
16:04 - Connection status: False
16:04 - Status: TibState.Standby
16:04 - Alarms: AlarmVector.NoAlarm
16:04 - Connection status: False
16:04 - Status: TibState.Armed
16:04 - Alarms: AlarmVector.NoAlarm
16:04 - Connection status: True
16:04 - Starting TIB triggers...
16:04 - Enabling trigger
16:04 - Status: TibState.Running
16:04 - Alarms: AlarmVector.NoAlarm
16:04 - Connection status: True
16:05 - Configuring calibration box...
16:05 - Configuring calibration box with leds=0x40 voltage=12.00 frequency=5000.0/1 central_current=0.0 duration=40 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=False enabled=False
16:05 - Starting flashes...
16:07 - Stopping TIB...
16:07 - Status: TibState.Running
16:07 - Alarms: AlarmVector.ReadError
16:07 - Connection status: True
16:07 - Disabling trigger
16:07 - Disabling PPS
16:07 - Status: TibState.PpsStopped
16:07 - Alarms: AlarmVector.ReadError
16:07 - Connection status: True
16:07 - Closing TIB connection to camera server
16:07 - Stopping TIB...
16:07 - Status: TibState.PpsStopped
16:07 - Alarms: AlarmVector.ReadError
16:07 - Connection status: False
16:07 - Disabling trigger
16:07 - Failed to call DisableTrigger: Bad statuses are present
16:07 - Stopping NMC
16:07 - Found 0 PPS errors.
16:07 - Resetting NMC
16:07 - Starting task "Configure NMC"
16:07 - Configuring modules with configuration /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
16:07 - NMC is in state NmcState.Configured and knows of 2 drawer(s)
16:07 - Found 0 PPS errors.
16:07 - Finished task "Configure NMC"
16:07 - Setting HVPA ramp sped to 20 V/s
16:07 - Switching HVPA to state Nominal
16:07 - Switching drawer 127 HVPA to state Nominal
16:07 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
16:07 - Switching drawer 128 HVPA to state Nominal
16:08 - Starting NMC
16:08 - Clearing PPS error counters
16:08 - Checking high voltages...
16:08 - Starting task "Check HV"
16:08 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
16:08 - Done
16:08 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
16:08 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 964.41 V and 1360.91 V.
16:08 - Maximum difference between expected and actual voltages in drawer 127 is 1161.52 V
16:08 - Maximum difference between expected and actual voltages in drawer 128 is 5.27 V
16:08 - Checking status of HVPA module in 2 drawer(s)...
16:08 - Checking status of HVPA channels in 2 drawer(s)...
16:08 - HV was locked in 0 channel(s), was in rise default in 0 channel(s) and was still in progress in 6 channel(s)
16:08 - Finished task "Check HV"
16:08 - Checking high voltages...
16:08 - Starting task "Check HV"
16:08 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
16:08 - Done
16:08 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
16:08 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
16:08 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
16:08 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
16:08 - Checking status of HVPA module in 2 drawer(s)...
16:08 - Checking status of HVPA channels in 2 drawer(s)...
16:08 - HV is stabilized in all channels.
16:08 - Finished task "Check HV"
16:08 - Checking high voltages...
16:08 - Starting task "Check HV"
16:08 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
16:08 - Done
16:08 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
16:08 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
16:08 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
16:08 - Maximum difference between expected and actual voltages in drawer 128 is 0.39 V
16:08 - Checking status of HVPA module in 2 drawer(s)...
16:08 - Checking status of HVPA channels in 2 drawer(s)...
16:08 - HV is stabilized in all channels.
16:08 - Finished task "Check HV"
16:08 - Checking high voltages...
16:08 - Starting task "Check HV"
16:08 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
16:08 - Done
16:08 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
16:08 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
16:08 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
16:08 - Maximum difference between expected and actual voltages in drawer 128 is 0.57 V
16:08 - Checking status of HVPA module in 2 drawer(s)...
16:08 - Checking status of HVPA channels in 2 drawer(s)...
16:08 - HV is stabilized in all channels.
16:08 - Finished task "Check HV"
16:08 - NMC is in state NmcState.Running and knows of 2 drawer(s)
16:08 - Found 0 PPS errors.
16:08 - Configuring TIB...
16:08 - Setting TIB forced mode to False
16:08 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
16:08 - Status: TibState.PpsStopped
16:08 - Alarms: AlarmVector.ReadError
16:08 - Connection status: False
16:08 - Entering set_external_delay
16:08 - Setting ExternalDelayLocal to 0.000 ns
16:08 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
16:08 - Entering set_external_delay
16:08 - Setting ExternalDelayCalibration to 76.000 ns
16:08 - Writing 76.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
16:08 - Entering set_external_delay
16:08 - Setting ExternalDelaySphe to 0.000 ns
16:08 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
16:08 - Setting TIB trigger mode to 0x44 = TriggerMask.SlowControl|Calibration
16:08 - Writing 68 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
16:08 - Writing 0 to Camera.TIB.InputsEnable.InputMask.InputMask_v
16:08 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
16:08 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
16:08 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
16:08 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
16:08 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
16:08 - Status: TibState.PpsStopped
16:08 - Alarms: AlarmVector.ReadError
16:08 - Connection status: False
16:08 - Writing False to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
16:08 - Writing 10000 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
16:08 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:08 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
16:08 - Writing 10000 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
16:08 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
16:08 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:08 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
16:08 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
16:08 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:08 - Connecting TIB to camera server at 10.10.4.12:222
16:08 - Starting TIB...
16:08 - Status: TibState.PpsStopped
16:08 - Alarms: AlarmVector.ReadError
16:08 - Connection status: False
16:08 - Resetting run
16:08 - Wait for UCTS external trigger before enabling trigger...
16:08 - Status: TibState.Standby
16:08 - Alarms: AlarmVector.NoAlarm
16:08 - Connection status: False
16:08 - Status: TibState.Standby
16:08 - Alarms: AlarmVector.NoAlarm
16:08 - Connection status: False
16:08 - Status: TibState.Armed
16:08 - Alarms: AlarmVector.NoAlarm
16:08 - Connection status: False
16:08 - Starting TIB triggers...
16:08 - Enabling trigger
16:08 - Status: TibState.Running
16:08 - Alarms: AlarmVector.NoAlarm
16:08 - Connection status: False
16:09 - Configuring calibration box...
16:09 - Configuring calibration box with leds=0x40 voltage=12.00 frequency=5000.0/1 central_current=0.0 duration=400 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=False enabled=False
16:10 - Stopping NMC
16:10 - Found 0 PPS errors.
16:10 - Resetting NMC
16:10 - Starting task "Configure NMC"
16:10 - Configuring modules with configuration /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
16:10 - NMC is in state NmcState.Configured and knows of 2 drawer(s)
16:10 - Found 0 PPS errors.
16:10 - Finished task "Configure NMC"
16:11 - Setting HVPA ramp sped to 20 V/s
16:11 - Switching HVPA to state Nominal
16:11 - Switching drawer 127 HVPA to state Nominal
16:11 - Failed to write 'NectarFrontEnd_127.Hvpa.SwitchVoltages': - overallStatus : Bad statuses are present - requestHandle : 7 - targets[] - targets[0] status: The server could not successfully process the Write service: BadInternalError: The server could not successfully process the Write service: BadInternalError - opcUaStatusCode : 2.14761e+09
16:11 - Switching drawer 128 HVPA to state Nominal
16:11 - Starting NMC
16:11 - Clearing PPS error counters
16:11 - Checking high voltages...
16:11 - Starting task "Check HV"
16:11 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
16:11 - Done
16:11 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
16:11 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
16:11 - Maximum difference between expected and actual voltages in drawer 127 is 1160.91 V
16:11 - Maximum difference between expected and actual voltages in drawer 128 is 0.57 V
16:11 - Checking status of HVPA module in 2 drawer(s)...
16:11 - Checking status of HVPA channels in 2 drawer(s)...
16:11 - HV is stabilized in all channels.
16:11 - Finished task "Check HV"
16:11 - Checking high voltages...
16:11 - Starting task "Check HV"
16:11 - Checking reference voltages against /local/home/cta_test_bench/config/2018/20181017/spe_config_2modules.NMC.xml
16:11 - Done
16:11 - Found 1 drawer(s) in Nominal state, 1 in Low state and 0 in Wait state
16:11 - Found 2 drawer(s). Reference voltage is between 966.45 V and 1360.86 V. Actual voltage is between 966.24 V and 1360.91 V.
16:11 - Maximum difference between expected and actual voltages in drawer 127 is 1160.30 V
16:11 - Maximum difference between expected and actual voltages in drawer 128 is 0.37 V
16:11 - Checking status of HVPA module in 2 drawer(s)...
16:11 - Checking status of HVPA channels in 2 drawer(s)...
16:11 - HV is stabilized in all channels.
16:11 - Finished task "Check HV"
16:11 - Configuring TIB...
16:11 - Setting TIB forced mode to False
16:11 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
16:11 - Status: TibState.Running
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: True
16:11 - Disabling PPS
16:11 - Error configuring TIB: Failed to call DisablePPS: Bad statuses are present
16:11 - Stopping TIB...
16:11 - Status: TibState.Running
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: True
16:11 - Disabling trigger
16:11 - Disabling PPS
16:11 - Status: TibState.PpsStopped
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: True
16:11 - Closing TIB connection to camera server
16:11 - Resetting TIB...
16:11 - Failed to call Reset: Bad statuses are present
16:11 - Configuring TIB...
16:11 - Setting TIB forced mode to False
16:11 - Writing False to Camera.TIB.Forced.ForcedMode.ForcedMode_v
16:11 - Status: TibState.RunStopped
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: False
16:11 - Disabling PPS
16:11 - Status: TibState.PpsStopped
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: False
16:11 - Entering set_external_delay
16:11 - Setting ExternalDelayLocal to 0.000 ns
16:11 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelayLocal.ExternalDelayLocal_v
16:11 - Entering set_external_delay
16:11 - Setting ExternalDelayCalibration to 76.000 ns
16:11 - Writing 76.0 to Camera.TIB.ExternalDelay.ExternalDelayCalibration.ExternalDelayCalibration_v
16:11 - Entering set_external_delay
16:11 - Setting ExternalDelaySphe to 0.000 ns
16:11 - Writing 0.0 to Camera.TIB.ExternalDelay.ExternalDelaySphe.ExternalDelaySphe_v
16:11 - Setting TIB trigger mode to 0x44 = TriggerMask.SlowControl|Calibration
16:11 - Writing 68 to Camera.TIB.TriggerMode.ModeMask.ModeMask_v
16:11 - Writing 0 to Camera.TIB.InputsEnable.InputMask.InputMask_v
16:11 - Writing False to Camera.TIB.BUSY.DisableBusy.DisableBusy_v
16:11 - Writing 0.0 to Camera.TIB.BUSY.ExtraDelayBusy.ExtraDelayBusy_v
16:11 - Writing False to Camera.TIB.Throttler.DisableThrottler.DisableThrottler_v
16:11 - Writing 255 to Camera.TIB.Throttler.MaxTriggers50us.MaxTriggers50us_v
16:11 - Writing 255 to Camera.TIB.Throttler.MaxTriggers1ms.MaxTriggers1ms_v
16:11 - Status: TibState.PpsStopped
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: False
16:11 - Writing False to Camera.TIB.Pedestal.PeriodicPedestalEnable.PeriodicPedestalEnable_v
16:11 - Writing 10000 to Camera.TIB.Pedestal.PeriodicPedestalFrequency.PeriodicPedestalFrequency_v
16:11 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:11 - Writing False to Camera.TIB.Pedestal.DigitalPedestalEnable.DigitalPedestalEnable_v
16:11 - Writing 10000 to Camera.TIB.Pedestal.DigitalPedestal_config_Rate.DigitalPedestal_config_Rate_v
16:11 - Writing 375764 to Camera.TIB.Pedestal.DigitalPedestalSeed.DigitalPedestalSeed_v
16:11 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:11 - Writing False to Camera.TIB.Pedestal.AnalogPedestalEnable.AnalogPedestalEnable_v
16:11 - Writing 3000.0 to Camera.TIB.Pedestal.AnalogPedestalThreshold.AnalogPedestalThreshold_v
16:11 - Writing False to Camera.TIB.Pedestal.Fake_trig_enable.Fake_trig_enable_v
16:11 - Connecting TIB to camera server at 10.10.4.12:222
16:11 - Starting TIB...
16:11 - Status: TibState.PpsStopped
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: False
16:11 - Resetting run
16:11 - Wait for UCTS external trigger before enabling trigger...
16:11 - Status: TibState.Standby
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: False
16:11 - Status: TibState.Standby
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: False
16:11 - Status: TibState.Armed
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: True
16:11 - Starting TIB triggers...
16:11 - Enabling trigger
16:11 - Status: TibState.Running
16:11 - Alarms: AlarmVector.NoAlarm
16:11 - Connection status: True
16:11 - Configuring calibration box...
16:11 - Configuring calibration box with leds=0x40 voltage=12.00 frequency=5000.0/1 central_current=0.0 duration=400 width=1 lemo_out=True fiber1_out=True fiber2_out=True lemo_in=False enabled=False
16:11 - Starting flashes...