Viewing alarms

When there are active alarms on your Pexip Infinity deployment, a flashing red icon appears at the top right of each page of the Administrator interface. To view details of the current alarms, click on this icon or go to the Alarms page (Status > Alarms).

  • Alarms remain in place for as long as the issue exists. After the issue has been resolved (for example, if a conference ends, therefore freeing up licenses) the associated alarm will automatically disappear from the Alarms page.
  • Multiple instances of the same type of alarm can be raised. For example if two Conferencing Nodes are not correctly synchronized to an NTP server, you will see an alarm for each node.
  • You can select individual alarms and view the associated documentation (this guide) for suggested causes and resolutions.

The Status > Alarm History page shows the details of all historic alarms including the severity level, and the time the alarm was raised and lowered.

An alarm is raised in each of the following situations:

Alarm ID Logged as Alarm = Level Cause Suggested resolutions
The Management Node does not have a TLS certificate 20

tls_certificate_missing_management

Critical The Management Node has no associated TLS certificate. Upload a TLS certificate and associate it with the Management Node.
A Conferencing Node does not have a TLS certificate 9

tls_certificate_missing

Critical A Conferencing Node has no associated TLS certificate.

Upload a TLS certificate and associate it with the Conferencing Node.

Alternatively, and if appropriate for your deployment, associate an existing certificate with your Conferencing Node. When doing this, the existing certificate should already contain a SAN (Subject Alternative Name) that matches your Conferencing Node's FQDN.

See Managing a node's TLS server certificate for more information.

CPU instruction set not supported 10

cpu_not_supported

Critical

A Conferencing Node has gone into maintenance mode because it was deployed on a server with an unsupported processor instruction set (e.g. SSE4.1).

This could also be caused by setting the EVC mode on a VMware cluster to too low a level, such as Westmere (see Enhanced vMotion Compatibility (EVC) for more information).

Deploy the Conferencing Node on a server with AVX or AVX2.
NTP not synchronized 11

ntp_not_synchronised

Error A node has failed to synchronize with the configured NTP servers. Ensure that NTP is enabled on the Management Node, and that NTP servers are assigned to each location. See Syncing with NTP servers for more information.
Configuration not synchronized 18

configuration_sync_failure

Error This alarm is raised if the Conferencing Node status “Last contacted” time has not been updated within the last 2 expected replication intervals (typically no contact within the last 3 minutes).

In typical deployments, configuration replication is performed approximately once per minute. However, in very large deployments (more than 60 Conferencing Nodes ), configuration replication intervals are extended, and it may take longer for configuration changes to be applied to all Conferencing Nodes (the administrator log shows when each node has been updated).

If configuration synchronization fails this may indicate network connectivity or routing issues between the Management Node and the Conferencing Node, which could be due to a malfunction or misconfiguration of devices such as routers or firewalls etc.

Ensure that all of the appropriate Pexip nodes are fully routable to each other in both directions. See General network requirements.

MS Exchange Connection Failure 22

scheduling_connection_failure

Error The Management Node cannot connect to the Exchange server. Check that the details entered in the EWS URL (System Configuration > Pexip Exchange Integrations) is correct and the Exchange server is online.
Automatic backup upload failed 25

autobackup_upload_failed

Error The Management Node cannot connect to the FTP server to upload a backup file. Check that the Upload URL (supported schemes are FTPS and FTP) and the Username and Password credentials of the FTP server are correct (Utilities > Automatic backups) and that the Management Node can reach the FTP server.
License limit reached 2

licenses_exhausted

Warning A Conferencing Node is unable to accept a call because there are not enough concurrent licenses available on the system at this time. For more information, see Pexip Infinity license installation and usage.
  • Wait until one or more of the existing conferences have finished and the licenses have been returned to the pool.
  • Contact your Pexip authorized support representative to purchase more licenses.

Note that when a license subsequently becomes available (e.g. because a participant leaves a conference, or because the administrator adds more licenses), the alarm is not cleared immediately; the alarm is cleared after the next participant successfully joins a conference.

Licenses expiring 3

licenses_expiring

Warning One or more of your licenses is due to expire within the next 60 days. Contact your Pexip authorized support representative to renew your licenses.
Call capacity limit reached 1

capacity_exhausted

Warning

A call has not been accepted because all Conferencing Nodes that are able to take the media for this call are at capacity.

Note: to understand how often this issue is occurring in your deployment, search the Administrator log for "insufficient capacity".

  • Deploy more Conferencing Nodes in this location.
  • Move existing Conferencing Nodes onto more powerful servers.
  • Allocate more virtual CPUs for Conferencing Nodes on existing servers (if there are sufficient CPU cores). Note that the Conferencing Node will have to be rebooted for this to take effect.
  • Configure each location with a primary and secondary overflow location.
  • If a call is received by a Conferencing Node that is in a location that only contains Proxying Edge Nodes, that location must be configured with a Transcoding location that contains your Transcoding Conferencing Nodes.

Note that some types of call consume more resources than other calls. Thus, for example, if you are at full capacity and an audio-only call disconnects, there may still not be sufficient free resource to connect a new HD video call. For further information on capacity and how calls consume resources, see Hardware resource allocation rules.

Management Node limit reached 5

management_node_exhausted

Warning The Management Node does not have sufficient resources for the current deployment size (number of Conferencing Nodes).

Increase the amount of RAM and the number of virtual CPUs assigned to the Management Node.

See the recommended hardware requirements in Server design recommendations.

Trusted CA certificates expiring 6

trustedca_expiring

Warning One or more of your trusted CA certificates is due to expire within the next 30 days, or has already expired. Obtain and upload an updated certificate for the certificate authority.
TLS certificates expiring 7

tls_certificate_expiring

Warning One or more of your TLS certificates is due to expire within the next 30 days, or has already expired. Obtain and upload an updated TLS certificate. You may also need to delete the old certificate.
Incomplete TLS certificate chains 8

tls_certificate_chains

Warning A TLS certificate has an incomplete chain of trust to the root CA certificate. Obtain and upload the appropriate chain of intermediate CA certificates to the Management Node (the certificate provider normally provides the relevant bundle of intermediate CA certificates).
Syslog server inaccessible 4

syslog_inaccessible

Warning A syslog server has been configured to use TCP or TLS but either is not responding to contact requests, or the connection has dropped.
  • Check your network connectivity.
  • Check that the syslog server is running.
Connectivity lost between nodes 19

connectivity_lost

Warning

Communication to a Pexip Infinity node has been lost.

When a connection is lost, Pexip Infinity tries to contact the node every 5 seconds until the connection is re-established. In large deployments with several connectivity failures, Pexip Infinity attempts to re-establish connections to a maximum of 10 nodes at a time.

Intermittent short-lived "Connectivity lost between nodes" alarms may be an indication of an unreliable network.

These alarms may be raised for a short period of time — and this is expected behavior — if a node is placed into maintenance mode and Proxying Edge Nodes need to establish new connectivity paths.

They may also occur during initial deployment or an upgrade, and this is also expected behavior. They automatically clear as each node is upgraded to the new software version, has restarted and is ready to handle calls.

Check network connectivity and routing as for "Configuration not synchronized" above, or in the case of a software upgrade, wait for the upgrade process to complete.
Hardware instability detected 21

irregular_pulse

Warning Pexip Infinity has detected that the underlying VM infrastructure has paused the Pexip virtual machine. This is usually indicative of over-committed hardware, which we do not support. Pexip Infinity, since it is a real time system, requires dedicated access to the underlying CPU and RAM resources of the hardware host.

Ensure that the Management Node and all Conferencing Nodes have dedicated access to their own RAM and CPU cores.

See the recommended hardware requirements in Server design recommendations.

CPU instruction set is deprecated 23

cpu_deprecated

Warning

The node is deployed on a server that is not using the AVX/AVX2 CPU instruction set (e.g. if it uses SSE4.2).

Support for the current instruction set on this server is deprecated, and in the future you will not be allowed to upgrade Pexip nodes running on servers with this instruction set.

This alarm is raised when a Conferencing Node restarts and is automatically cleared after 48 hours.

Deploy the Conferencing Node on a server with AVX or AVX2.
Hardware IO (input/output) instability detected 24 io_high_latency Warning Pexip Infinity has recently detected consistent read latency greater than 150ms or write latency greater than 200ms.
  • Avoid having multiple VMs using the same physical hard drive.
  • Check the hard drive for failures.

Cloud bursting alarms

The following alarms may be raised in relation to issues with dynamic cloud bursting. See Dynamic bursting to a cloud service for more information about resolving these alarms.

Alarm ID Logged as Alarm = Level Cause Suggested resolutions
Not authorized to perform this operation

15

&

16

bursting_unauthorized_instance_failure

bursting_unauthorized_region_failure

Error Pexip Infinity is not authorized to view instance data or to start and stop instances in the cloud service.

For AWS, ensure that an appropriate policy document is configured in AWS and is attached to the user that is being used by the Pexip platform.

For Azure, check your Active Directory (AD) application and its associated role/permissions.

Authentication failure while trying to communicate with the cloud provider 17

bursting_authentication_failure

Error Pexip Infinity cannot sign in to the cloud service.

For AWS, check that the Access Key ID and Secret Access Key that are configured in Platform configuration > Global settings match the User Security Credentials for the user you added within Identity and Access Management in the AWS dashboard.

For Azure, check that your subscription, client and tenant IDs and secret key are correct for your Active Directory application.

Cloud bursting process encountered an unexpected error 12

bursting_error

Error Pexip Infinity encountered an unexpected error while managing the cloud overflow nodes.

Check the status of your cloud bursting nodes within Pexip Infinity (Status > Cloud bursting) and of your instances within your cloud provider.

Also check administrator and support log messages that are tagged with a log module name of administrator.alarm to see additional error message information.

Cloud-bursting node found, but no corresponding Conferencing Node has been configured 13

bursting_missing_pexip_node

Warning

This occurs when Pexip Infinity detects a bursting instance with a tag matching your system's hostname but there is no corresponding Conferencing Node configured within Pexip Infinity.

This message can occur temporarily in a normal scenario when deploying a new Conferencing Node and you have set up the VM instance in your cloud provider but you have not yet deployed the Conferencing Node in Pexip Infinity. In this case, the issue will disappear as soon as the Conferencing Node is deployed.

A location contains cloud bursting nodes, but no other locations are using it for overflow 14

bursting_no_location_overflow

Warning A location contains some cloud overflow nodes, but no other locations are using it as an overflow location. Set the location containing the cloud overflow nodes as the Primary overflow location of the locations containing your "always on" Conferencing Nodes.