By Young Hoo Kim (Senior Software Engineer) and Therese Lee (Senior Software Engineer), XL8 Inc.

Oct 31, 2022

How familiar are you with subtitle formats? XL8 is the leader in AI-powered machine translation for subtitled media. Although I am a Software Engineer at XL8, my knowledge on subtitle formats has been limited, except the most widely used subtitle format called SRT. However, I often get questions such as:

As a software engineer, I can tell that my answers will depend on which subtitle format you’re using. To find out more accurate answers and also for my own knowledge, I did some study on the subtitle formats used by XL8 and the media and translation industries. The outcome of my research was quite surprising.

Frankly speaking, I believe that most engineers are already familiar with a world of technical fragmentation where many specifications and proposals are competing with each other to become the de facto standards in the field. For example, A1, which is a successor of A, complements something but lacks backward compatibility, thus creating a great deal of confusion to its users. But this chaos is “usual” to software engineers so they stay sane and live with them.

When it comes to subtitle formats, however, it was more difficult to understand all of them at once because many of them were created without a purpose to become a standard; and overall, it was difficult to comprehend the reason behind why a new format was proposed or which particular features have been enhanced. Some formats do not have any specification documents. Most importantly, many of these formats exclusively support only their specific applications and have no intention to compete with other formats.

Before beginning this research, I presumed that I would have to scrape up many wiki pages, but it ultimately turned out that many of the formats lack wiki pages. In this blog post, I’m going to highlight the most notable findings of my research. As I previously mentioned, I did my best in trying to gather meaningful information from this barren field.

SRT is one of the formats that does have a wiki page. It is named SubRip in the wiki page because it was developed for “SubRip,” a free Windows software released in 2000 using Delphi. SRT is an abbreviation of SubRip Text. The format's wiki page described how it got its name. It was developed for "SubRip," a free Windows software released in 2000 using Delphi.

origin: SubRip of Softpedia

You might have seen the content of the SRT file at least once.

The SRT file consists of the following components:

Let’s redirect our thinking now back to one of the questions: Is it possible to change the color of the text? Since SubRip doesn’t explicitly address this feature, the ability to change the text color depends on the media player presenting the subtitles. The majority of players support a few html style tags like <b> for bold, <i> for italic, <u> for underline and <font color=”#code”> for font color.

Since it is unofficial (no “official” document exists), the correct answer to the question is, “It depends on your media player. You should test it out on the player that you want to use and determine whether it displays the color correctly or not.” In the worst case, it may display the raw html tag to the viewers.

How about the position of the text? Can we adjust its position on the screen? Again, it is possible unofficially using ASS, and this capability depends on the specific media players used. I have even seen subtitles with tags like “{\an8}” on a player that doesn’t support those unofficial tags.

Many subtitle formats including SRT are for specific applications, so there are no standards and specifications. That’s why the term “unofficial” is a little bit uncomfortable. However, the TTML subtitles described below differ from most subtitle formats becuase they contain a particular set of specifications.

TTML (Timed Text Markup Language), previously termed DFXP, has the W3C standard and uses XML, a markup language familiar to engineers.

This format stipulates the styling and alignment of subtitles. It can alter the subtitle text’s color and alignment in many different ways like the picture below.

Although XML is a bit lengthy, many media players support this format. Moreover, it is one of the most widely used subtitle formats. TTML is one of the best options when advanced features are needed.

In 2010, when discussions of HTML5 were actively occurring, debates swirled surrounding the standard of subtitles. One of the debates was which subtitle standard would be used with the <track> tag for the <video> elements. W3C considered SRT by SubRip and TTML using the XML standard, but ultimately neitherwas selected. The new subtitle format, called WebVTT, was inspired by SRT. Although the specification of WebVTT is still in its preliminary phase, the vast majority of browsers support this format.

WebVTT is far more familiar because it’s based on SRT. Additionally, it supports a standard of style or alignment.

Also it is one of few formats which specifies the encoding as UTF8. Most other subtitle formats do not specify the encoding; therefore, when the file needs to be parsed, charset detection has to be performed.

SCC is the abbreviation for Scenarist Closed Caption, widely used for Adobe Premiere or Final Cut Pro. You might know of Closed Caption if you explore the world of subtitles. The Closed Caption is not a burned-in subtitle; in other words, it is the raw subtitle in the file format as is. Because it is not burned-in yet, it has the option of being presented on the screen or not. The burned-in subtitles are called Open Caption, Burned-in Caption or Baked on Caption.

The SCC format has several features for text styling and alignment. However, it expresses text as 2bytes hexadecimal, and supports a very limited character set. As a result, lots of subtitles that consist of the languages which don't use the general Latin alphabet like Korean, Chinese, Japanese, etc. can’t use this format. It also has limitations of the frame rate (29.97) and length of the text in a line (37).

STL is the abbreviation for Spruce Subtitle File, developed by Spruce Technologies Inc. for their application — DVD Studio Pro. STL is not widely used but requires mentioning due to its relationship with EBU STL which is discussed next.

This version of STL was developed by European Broadcasting Union (EBU) and uses the same file extension with the STL by Spruce Technologies Inc. Unfortunately, subtitle creators are required to verify whether the format is EBU STL or Spruce STL, which creates more work. Contrary to the other formats described so far, which use plain text, the EBU format consists of binary data. Thus, it can’t be read with a general text editor.

PAC is also a binary format, developed by Screen Subtitling Systems. It supports text styling and alignment, but it's really hard to find the specification.

XIF is the subtitle format based on XML, and it is also one of the file formats that does not have specifications available.

SAMI is an abbreviation for Synchronized Accessible Media Interchange, which is similar to the HTML form. Originally, Microsoft developed this format to improve accessibility for media players, and SASAMI 2K, a media player developed by Koreans, supported it. Since the player enjoyed extraordinary popularity in Korea, the SMI format became the most popular one in the country as if it was the standard.

Many people mistake the file format as developed for SASAMI (Specially Advanced Synchronized Accessible Media Interchange) because the name looks very similar to SAMI or SMI. But, Microsoft left it as is without any updates or additional support. Moreover, the Movie & TV application of Windows 10 doesn’t  support the SMI format.

Origin: 네이버 포스트 - 동영상 플레이어의 변천사

SMI has features for styling and alignment. However, it might be better to use others for compatibility because many media players do not support this format.

During this research, it was hard to find specifications for many formats and to understand the purpose for which a format was developed. There are so many “Couldn’t find a specification for this format” on the WHATWG page. In my opinion, it seems that this is because most formats were developed for a specific application and no effort was made for these formats to be standard except TTML and VTT. However, this is just speculation.

Although XL8 supports most formats that are mentioned above, most clients use SRT, TTML and VTT formats. If more advanced features are required, you can use TTML or VTT. However, I don’t think the other formats are needed because of their archaic nature. Plenty of engineers have suffered and are suffering from using these archaic formats to support legacy system, even though it's 2022! My hat goes off to them for their heroic efforts and all their hard work.