Enabling live captions and transcripts

If your deployment is integrated with Pexip's AI Media Server (AIMS), you can enable speech-to-text transcription for meetings in Virtual Meeting Room and Virtual Auditoriums.

Using transcription

There are two ways in which transcription can be enabled and the transcribed text made available. You can use either or both at the same time:

  • Provided to participants as live captions text, either in the original spoken language, or translated into a language of their choosing.

  • Sent to an event sink as a series of transcripts for further processing.

Live captions

When live captions are supported in a meeting, each Pexip web app participant has the option to enable captions. When a participant has enabled live captions, all other participants are advised that the meeting is being transcribed, but the live captions are only shown to the participant(s) who have enabled them. By default the captions are shown in the same language being spoken in the meeting, but participants can elect to view their own captions in a different language. The captions are ephemeral — each participant will only see the captions generated since they joined the meeting, and they cannot save the captions.

Event sink transcripts

When transcripts are enabled in a supported meeting, each block of meeting audio is transcribed and sent as an event to an event sink. Transcripts are in the same language as that spoken in the meeting, but if a participant has enabled live captions in another language, the translated caption shown to the participant will also be sent to the event sink.

Using transcripts and live captions together

Live captions and transcripts are independent. If event sink transcripts are enabled for a meeting they will be sent even if no participants have enabled live captions; and when live captions are in use, transcripts are not sent to the event sink unless the transcript option is also enabled.

Security considerations

Pexip Infinity does not log or retain the contents of any live captions or transcripts. When transcripts are enabled, they are sent directly to the event sink from the Conferencing Node — they do not get sent to the Management Node.

Participants will always be made aware that transcripts or live captions have been enabled, via both an audio announcement and an on-screen icon.. Note that participants who join a meeting after live captions or transcripts have been enabled will not hear the audio announcement, but will still see the icons (if they are receiving video).

The transcripts sent to the event sink may contain confidential or sensitive data, which may be subject to GDPR or other legislation regarding the handling of PII. You must therefore ensure that you:

  • Restrict who can access the Management Node in order to create and configure event sinks and enable transcripts on VMRs (this can be done by restricting the May modify vmr configuration and May modify system configuration permissions for Administrators).
  • Protect the data once it is received by the event sink.
  • Ensure that data stored in the event sink is anonymised where required.

Enabling live captions support

In order to support live captions in your deployment, you must first:

  1. Configure and enable an AIMS integration.
  2. Enable transcription support in AIMS.
  3. Enable live captions support on VMRs.

After support is enabled, for an individual participant to receive live captions, either:

  • Pexip web app participants can use the Show live captions toolbar control to show live captions to themselves, or

  • API participants can use the show_live_captions request to show live captions to a specified participant.

Enabling live captions on VMRs and other services

Live captions are disabled by default. You can change the availability of live captions globally, and override this on a per-service basis.

To set the default availability of live captions for all services:

  1. From your Pexip Infinity Management Node, go to Platform > Global Settings.
  2. In the Service configuration section, select or clear the Enable live captions by default checkbox.

To override the global setting for an individual service (such as a VMR):

  1. From Services > Virtual meeting rooms select the VMR.
  2. Scroll down to and expand the Advanced Options section. From the Live Captions drop down, select one of the following options:
    • Use global live captions setting: live captions are only available for this VMR if the Platform > Global Settings > Service configuration > Enable live captions by default checkbox is selected.
    • Yes: live captions are always available for this VMR.
    • No: live captions are not available and cannot be enabled by participants in this VMR.

Enabling transcript support

In order to send transcripts to an event sink, you must first:

  1. Configure and enable an AIMS integration.
  2. Enable transcription support in AIMS.
  3. Configure an event sink in Pexip Infinity. The Events to send must include livecaptions_data.
  4. Enable transcriptions on a VMR.

After support is enabled, for transcripts to be sent from a VMR or Virtual Auditorium to the event sink, either:

  • the Transcript option must be set to Best effort before the meeting starts, or

  • The Transcript option must be set to On demand and an API participant must then send the start_transcript request.

Enabling transcripts on VMRs and other services

By default, transcripts are enabled globally On demand, meaning that transcripts will only be sent to an event sink if your deployment is enabled to support transcripts, and a participant specifically requests them. Transcripts are configurable on a platform-wide and per-conference basis.

To set the default availability of transcription for all services:

  1. Go to Platform > Global settings.
  2. From within the Service configuration section, select one of the following Transcript options:

    • Best effort: transcripts are sent to an event sink for all meetings (if your deployment has been configured to support transcripts).
    • On demand: transcripts are sent to an event sink only if they are enabled for that meeting using the start_transcript API command.
    • Disallowed: transcripts are never sent to an event sink.

To override the global setting for an individual service (such as a VMR):

  1. Go to Services > Virtual Meeting Rooms or Services > Virtual Auditoriums and select the service.
  2. From within the Advanced options section, choose one of the Transcript options described above.