Microsoft brings live transcription to Teams

Microsoft brings live transcription to Teams

When you need a written record of what people said

Since March 2021, the transcription feature has been available. The availability of the transcript complements the task of creating a meeting summary. It highlights important information shared during calls and adds to a series of recent improvements in Teams.

Transcription is available for Office/Microsoft 365 E3 and E5 plans and Microsoft 365 Business Standard and Premium plans. Currently, transcription works for personal meetings. It is not available for channel meetings, Meet Now meetings, or 1:1 calls. Meeting transcription is only available in the Teams desktop client.

Update: According to Message Center notification MC260564 (June 7), Microsoft is making transcription available for these additional SKUs to make Teams meeting content more accessible to people with hearing impairments:

• Office 365 E1
• Office 365 A1
• Microsoft 365 Business Basic
• Office 365/Microsoft 365 A3
• Office 365/Microsoft 365 A5
• Microsoft 365 F1
• Office 365/Microsoft 365 F3

Meeting transcription (automatic captions) is a Stream feature. Meeting recording storage has been moved to OneDrive for Business, so the ability to generate transcripts has been lost. This gap no longer exists, which is a relief. By default, OneDrive stores all new Teams meeting recordings.

Change in plan

A notable change since Microsoft’s initial release (MC220987) is that recording is no longer combined with transcription. Two separate options appear in the Teams meeting menu (Figure 1). You can start recording without generating a transcript or start transcription without recording a meeting. Separate notifications are used to inform meeting participants when recording and transcription are active.

Transcription Policy Control

To transcribe a meeting, the AllowTranscription setting must be active in the Teams meeting policy. Only PowerShell can change the AllowTranscription setting. To check the global (default) policy settings, run the Get-CsTeamsMeetingPolicy cmdlet (included in the Teams PowerShell module):

Get-CsTeamsMeetingPolicy -Identity Global | ft identity, AllowTranscription, AllowCloudRecording
 
Identity                    AllowTranscription AllowCloudRecording
--------                    ------------------ -------------------
Global                      False              True

To update the policy, run the Set-CsTeamsMeetingPolicy cmdlet:

Set-CsTeamsMeetingPolicy -Identity Global -AllowTranscription $True

Capturing speech during meetings

Transcription is the generation of an AI recording of the conversation during a meeting. Processing occurs in near real-time to display the text during meetings. And to make the full transcript available shortly after the meeting ends.

Use of AI

In its blog post, Microsoft states that “Delivering live transcription with high accuracy, minimal latency, and cost-effectiveness at enterprise scale has been one of the toughest challenges in the industry. Over the past two years, we’ve made significant progress solving this problem and have significantly improved our accuracy models using real-time meeting context and cutting-edge AI.”

Microsoft also notes that an AI model is created for each meeting to consider the meeting topic, attendees, and attachments to improve text recognition accuracy, especially around jargon. Microsoft notes that the models used for meeting transcription are deleted after the meeting ends and are not used to improve their AI.

There’s no doubt that capturing and transcribing meeting conversations is a dauntingly difficult computational task. The computational challenges will become even more difficult as Microsoft expands coverage to languages ​​other than US English. It’s currently the only language covered.

I’m not sure how Teams detects American English, as it seems to have no trouble interpreting my Irish accent. It might be more accurate to say that transcription works when people speak American-style transatlantic English and won’t work if someone switches to French, German, Italian, or another language.

Boundaries

A look at the transcript shown in Figure 2 shows how difficult it can be to capture an accurate transcript. If the meeting is full of different voices, you’ll see oddities in the transcript. Noise suppression in the Teams desktop client seems to help, as I’ve noticed better results when people use the desktop client rather than the browser or mobile clients.

Using Meeting Transcripts

When a meeting ends, the meeting transcript is available almost immediately. Open the meeting in the Teams calendar app, and the transcript is available in the meeting summary or via the Recordings & Transcripts tab. Figure 3 shows an example transcript of a meeting that took place in three parts, one for each segment. In this case, the meeting organizer is reviewing the transcript, so the delete option is available. Other participants can view and scroll down the transcripts, but they cannot delete a transcript.

Transcripts are available for download in Video Text Track (VTT) or Microsoft Word (docx) format. VTT files are text-based files where individual contributions are noted with a detailed time stamp. For example:

00:03:02.870 --> 00:03:05.570
<v Sean Landy>So now we get its goal.</v>

The Word format is more readable because it’s less structured and more valuable because the transcript is easier to edit. In both cases, the content corresponds exactly to what was displayed during the meeting. The availability of downloaded files allows you to edit obvious errors and add missing content before publishing the official meeting minutes.

Guest users don’t have access to the Teams calendar app, so they can’t access the transcript after the meeting ends.

Privacy

The transcription’s default is to use speaker attribution. In other words, when people speak, their names identify their contributions in the transcription. Users can mask their identities for live transcriptions and captions in the Captions & Transcriptions section of the client settings (Figure 4).

When a user chooses not to identify themselves in transcripts, Teams inserts the generic attribution “Speaker” with a number, so the transcript contains text for “Speaker 1”, “Speaker 2”, etc.

Also, caching means that enabling the option to allow/deny identification in transcripts is deferred.

Transcriptions and Compliance

Microsoft says, “Teams Live transcript files are stored in the meeting organizer’s Exchange Online account, and only the organizer and tenant administrator have permission to delete them.” In this case, let’s see where the transcript information is stored. After that, I used PowerShell to detect recent updates to the folders in my Exchange Online mailbox. Then, I used the MFCMAPI utility to examine the items in those folders. This shows that Teams stores transcript items in the ApplicationDataRoot/93c8660e-1330-4e40-8fda-fd27f9eafe10/MeetingTranscriptCollection folder. This folder is located in the NonIPMRoot (hidden part) folder of the mailbox. Each item refers to a transcript. If we examine the stored properties for the transcript, we find JSON content (Figure 5).

Then by pasting the JSON into a PowerShell variable, we can see what it contains:

$json | convertfrom-json
 
TranscriptJsonBlob@is.Large         : True
TranscriptJsonBlob@stream           : True
TranscriptJsonBlob@type             : Edm.String
TranscriptMetadataJsonBlob@is.Large : True
TranscriptMetadataJsonBlob@stream   : True
TranscriptMetadataJsonBlob@type     : Edm.String
TranscriptWordBag@is.Queryable      : False
TranscriptWordBag                   :
TranscriptWordBag@type              : Edm.String
ThreadId@is.Queryable               : True
ThreadId                            : 19:meeting_NWY0ZDMwZGMtZWVjYi00ZDFmLTlhYmMtNWYzOWU2NGVlY2Jh@thread.v2
ThreadId@type                       : Edm.String
MessageId@is.Queryable              : True
MessageId                           : 0
MessageId@type                      : Edm.String
#type.ThreadId                      : String
#type.MessageId                     : String

There’s no transcript text. Instead, we have the meeting thread link. This takes us to the Teams data store in Azure. A little more digging revealed a second property called TranscriptJsonBlob. This is where the transcript text is stored.

If Exchange Online mailboxes stored transcript text, the transcripts would be indexable and discoverable. And that’s exactly what will happen when Microsoft delivers the documented update to make transcripts searchable.

A promising new feature

Overall, there’s a lot to like about meeting transcription. Some people get better results than others. But experience will guide people toward using better setups when transcription is important. Automated transcripts can be imperfect and include interesting but incorrect interpretations of the conversation. However, having a transcript generated by Teams is much less expensive than having it done by a professional transcriber. Ultimately, the result is a good starting point for a final recording. Errors are easily detected by editing the transcript and making that version available to participants.