Subtitle (captioning)

Subtitles are textual versions of the dialog in films, television programs and (in recent years) video games, usually displayed at the bottom of the screen. They can either be a form of written translation of a dialog in a foreign language, or a written rendering of the dialog in the same language, with or without added information to help viewers who are deaf and hard-of-hearing to follow the dialog, or people who cannot understand the spoken dialogue or who have accent recognition problems. Sometimes, mainly at film festivals, subtitles may be shown on a separate display below the screen, thus saving the film-maker from creating a subtitled copy for perhaps just one showing. Television subtitling for the deaf and hard-of-hearing is also referred to as closed captioning in some countries. More exceptional uses also include operas.

Creation, delivery and display of subtitles

Besides creating the subtitles, the subtitler usually also tells the computer software the exact positions where each subtitle should appear and disappear. The end result is a subtitle file containing the actual subtitles as well as position markers indicating where each subtitle should appear and disappear. These markers are usually based on timecode if it is a work for electronic media (e.g., TV, video, DVD), or on film length (measured in feet and frames) if the subtitles are to be used for traditional cinema film. The finished subtitle file is used to add the subtitles to the picture, either : directly into the picture (open subtitles); embedded in the vertical interval and later superimposed on the picture by the end user with the help of an external decoder or a decoder built into the TV (closed subtitles on TV or video); or converted (rendered) to tiff or bmp graphics that are later superimposed on the picture by the end user's equipment (closed subtitles on DVD or as part of a DVB broadcast). For multimedia-style Webcasting, check: SMIL Synchronized Multimedia Integration Language; Timed Text DFXP.

Same language captions

Same language captions, i.e., without translation, were primarily intended as an aid for people who are deaf or hard-of-hearing. Internationally, there are several major studies which demonstrate that same-language-captioning can have a major impact on literacy and reading growth across a broad range of reading abilities. This method of subtitling is used by national television broadcasters in China and in India.

Same-Language-Subtitling
Same Language Subtitling (SLS) is the use of Synchronized Captioning of Musical Lyrics (or any text with an Audio/Video/ source) as a Repeated Reading activity. The basic reading activity involves students viewing a short subtitled presentation projected onscreen, while completing a response worksheet. To be really effective, the subtitling should have high quality synchronization of audio and text, and better yet, subtitling should change color in syllabic synchronization to audio model, and the text should be at a level to challenge students' language abilities.

Closed captions
Closed captioning is the American term for closed subtitles specifically intended for people who are deaf and hard-of-hearing. These are a transcription rather than a translation, and usually contain descriptions of important non-dialog audio as well such as "(sighs)" or "(door creaks)" and lyrics. From the expression "closed captions" the word "caption" has in recent years come to mean a subtitle intended for the hard of hearing, be it "open" or "closed". In British English "subtitles" usually refers to subtitles for the hard-of-hearing (HoH); however, the term "HoH subtitles" is sometimes used when there is a need to make a distinction between the two. Realtime Programs such as news bulletins, current affairs programs, sport, some talk shows and political events utilize realtime

or online captioning. Live captioning is increasingly common, especially in the United Kingdom and the United States, as a result of regulations that stipulate that virtually all TV eventually must be accessible for people who are deaf and hardofhearing. Pre-prepared Some programs may be prepared in their entirety several hours before broadcast, but with insufficient time to prepare a timecoded caption file for automatic play-out. Pre-prepared captions look very similar to offline captions, although the accuracy of cueing may be compromised slightly as the captions are not locked to program timecode. In the United States and Canada, some broadcasters have used it exclusively and simply left uncaptioned sections of the bulletin for which a script was unavailable. Newsroom captioning limits captions to pre-scripted materials and, therefore, does not cover 100% of the news, weather and sports segments of a typical local news broadcast which are typically not pre-scripted, last second breaking news or changes to the scripts, ad lib conversations of the broadcasters, emergency or other live remote broadcasts by reporters in-the-field. Live Communication Access Real-Time Translation (CART) stenographers, who use a computer with either stenotype or Velotype keyboards to transcribe stenographic input for presentation as captions within 23 seconds of the representing audio, must caption anything which is purely live and unscripted, however, the most recent developments include operators using speech recognition software (in the UK about 50% of all live captioning is through speech recognition). Realtime captions look different from offline captions, as they are presented as a continuous flow of text as people speak. Hybrid Because different programs are produced under different conditions, a case-by-case basis must consequently determine captioning methodology. Some bulletins may have a high incidence of truly live material, or insufficient access to video feeds and scripts may be provided to the captioning facility, making stenography unavoidable. Other bulletins may be pre-recorded just before going to air, making pre-prepared text preferable. Offline For non-live, or pre-recorded programs, television program providers can choose offline captioning. Captioners gear offline captioning toward the high-end television industry, providing highly customized captioning features, such as pop-on style captions, specialized screen placement, speaker identifications, italics, special characters, and sound effects. Offline captioning involves a five-step design and editing process, and does much more than simply display the text of a program. Offline captioning helps the viewer follow a story line, become aware of mood and feeling, and allows them to fully enjoy the entire viewing experience. Offline captioning is the preferred presentation style for entertainment-type programming.

SDH
"SDH" is an American term the DVD industry introduced. It is an acronym for "Subtitles for the deaf and hard-of-hearing", and refers to regular subtitles in the original language where important non-dialog information has been added, as well as speaker identification, useful when the viewer cannot otherwise visually tell who is saying what. The only significant difference for the user between "SDH" subtitles and "closed captions" is their appearance: SDH subtitles usually are displayed with the same proportional font used for the translation subtitles on the DVD; however, closed captions are displayed as white text on a black band, which blocks a large portion of the view. Many SDH subtitles present all of the text centered, while closed captions usually specify position on the screen: centered, left align, right align, top, etc. This is very helpful for speaker identification and overlapping conversation.

Use by those not deaf or hard-of-hearing

Although same-language subtitles and captions are produced primarily with the deaf and hard-of-hearing in mind, many hearing film and television viewers choose to use them. This is often done because the presence of closed captioning and subtitles ensures that not one word of dialogue will be missed. Bars and other noisy public places often make closed captions visible for patrons where dialogue would otherwise be drowned out. It is argued that such additional information and detail will enhance the overall experience and allow the viewer a better grasp on the material. Furthermore, people learning a foreign language may sometimes use same-language subtitles to better understand the dialog.

Translation
It's the quickest and the cheapest method of translating content, and is usually praised for the possibility to hear the original dialog and voices of the actors.

Subtitling
Realtime Realtime translation subtitling, usually involving simultaneous interpreter listening to the dialog quickly translating, while a stenographer types, is rare. The unavoidable delay, typing errors, lack of editing, and high costs regard very little need for translation subtitling. Allowing the interpreter to directly speak to the viewers is usually both cheaper and quicker, however, the translation is not accessible to people who are deaf and hardofhearing. Offline Some subtitlers purposely provide edited subtitles or captions to match the needs of their audience, for learners of the spoken dialog as a foreign language and for people with learning and/or mental disabilities. Thus, some programs display standard captions representing speech the program audio, word-for-word, if the viewer selects CC1 by using the television remote control or on-screen menu, however, they also provide edited captions of simplified sentences at a slower rate, if the viewer selects CC2.

Subtitles vs. dubbing and lectoring

The two alternative methods of 'translating' films in a foreign language are dubbing, in which other actors record over the voices of the original actors in a different language, and lectoring, a form of voice-over for fictional material where a narrator tells the audience what the actors are saying while their voices can be heard in the background. Lectoring is common for television in Russia, Poland, and a few other East European countries, while cinemas in these countries commonly show films dubbed or subtitled. Some DVD films include the simultaneous option of both subtitles and dubbing. In such cases, the translations are made separately, rather than the subtitles being a verbatim transcript of the dubbed scenes of the film.

Categories
Subtitles in the same language on the same production can be in different categories: Narrative This is the most common type of subtitle. Narrative subtitles are those in which spoken dialogue is displayed. These are most commonly used to translate a film with one spoken language and the text of a second language. Forced These are common on movies. Forced subtitles only provide subtitles when the characters speak a foreign or alien language, or a sign, flag, or other text in a scene is not translated in the localization and dubbing process. However, foreign dialogue may sometimes be left untranslated if the movie is meant to be seen from the point of view of a particular character who does not speak the language in question. Content They add content dictation that is missing from filmed action or dialogue. Content subtitles also appear in the beginning of some higher-budget films (e.g. Star Wars) or at the end of a film. Titles only Dubbed programs use this sort of subtitle. Titles only provide only the text for any untranslated on-screen text. They are most commonly forced (see above). Bonus Bonus subtitles are an additional set of text blurbs that are added to DVDs. Often shown in popup or balloon form, they point out humorous blunders in the filming or background/behind-the-scenes information to what is appearing on screen. Localized Localized subtitles are a separate subtitle track that uses expanded references (i.e. "The sake [a Japanese Wine] was excellent as was the Wasabi") or can replace the standardized subtitle track with a localized form replacing references to local custom (i.e. from above, "The wine was excellent as was the spicy dip").

Types
Subtitles exist in two forms; open subtitles are 'open to all' and cannot be turned off by the viewer; closed subtitles are designed for a certain group of viewers, and can usually be turned on/off or selected by the viewer. Hard (also known as hardsubs or open subtitles). The subtitle text is irreversibly merged in original video frames, and so no special equipment or software is required for playback. Hence, very complex transition effects and animation can be implemented, such as karaoke song lyrics using various colors, fonts, sizes, animation (like a bouncing ball) etc. to follow the lyrics. However, these subtitles cannot be turned off. Prerendered (also known as closed) subtitles are separate video frames that are overlaid on the original video stream while playing. Prerendered subtitles are used on DVD and Blu-ray (though they are contained in the same file as the video stream). It is possible to turn them off or have multiple language subtitles and switch among them, but the player has to support such subtitles to display them. Also, subtitles are usually encoded as images with minimal bitrate and number of colors. Changing such subtitles is hard, but special OCR software exists to convert such subtitles to "soft" ones. Soft (also known as softsubs or closed subtitles) are separate instructions, usually a specially marked up text with time stamps to be displayed during playback. It requires player support and, moreover, there are multiple incompatible (but usually reciprocally convertible) subtitle file formats. Softsubs are relatively easy to create and change. Text rendering quality can vary depending on the player, but is generally higher than prerendered subtitles. In other categorization, digital video subtitles are sometimes called internal, if they are embedded in a single video file container along with video and audio streams, and external if they are distributed as separate file (that is less convenient, but it is easier to edit/change such file).

Comparison table
Feature Can be turned off/on Multiple subtitle variants (for example, languages) Editable Player requirements None Hard No Prerendered Yes Soft Yes

Yes, though all displayed at the same time

Yes

Yes

No

Difficult, but possible Majority of players support DVD subtitles

Yes Usually requires installation of special software, unless national regulators mandate its distribution Low to High, depends on player and subtitle file format Depends on player and subtitle file format, but generally poor

Visual appearance, colors, font quality Transitions, karaoke and other special effects Distribution

Low to High, depends on video resolution/compression

Low

Highest

Low

Inside original video

Separate low-bitrate Relatively small subtitle file or video stream, commonly instructions stream, multiplexed multiplexed or separate High Low

Additional overhead

None, though subtitles added by re-encoding of the original video may degrade overall image quality, and the sharp edges of text may introduce artifacts in surrounding video

Subtitle formats
For software video players
Name Extension

Comparison table
Type Text Styling No Yes No Yes No Yes No No Yes Yes Yes No Yes Yes No Yes N/A N/A Metadata Timings Timing Precision

AQTitle JACOSub [11] MicroDVD MPEG-4 Timed Text

.aqt .jss .sub .ttxt (or mixed with A/V stream) .sub N/A (mixed with audio/video stream) .pjs .psb .rt .smi .ssf

Text Text Text XML

No No No No Yes Yes No No No Yes Yes No Yes Yes Yes Yes N/A N/A

Framings Elapsed Time Framings Elapsed Time

Dependent on Frame 10 Milliseconds Dependent on Frame 1 Millisecond

MPSub Ogg Writ

Text Text

Sequential Time Sequential Granules

10 Milliseconds Dependent on Bitstream Dependent on Frame 1 Second 10 Milliseconds Dependent on Frame 1 Millisecond

Phoenix Subtitle PowerDivX RealText [12] SAMI Structured Subtitle Format SubRip Gloss Subtitle (Advanced) SubStation Alpha SubViewer Universal Subtitle Format VobSub XSUB

Text Text HTML HTML XML

Framings Elapsed Time Elapsed Time Framings Elapsed Time

.srt .gsub .ssa or .ass (advanced)

Text HTML/XML Text

Elapsed Time Elapsed Time Elapsed Time

1 Millisecond 10 Milliseconds 10 Milliseconds

.sub .usf

Text XML

Elapsed Time Elapsed Time

10 Milliseconds 1 Millisecond

.sub + .idx N/A (embedded in .divx container) .stl

Image Image

Elapsed Time Elapsed Time

1 Millisecond 1 Millisecond

Spruce subtitle format [13]

Text Yes Yes

Sequential Time+Frames

Sequential Time+Frames

There are still many more uncommon formats. Most of them are text-based and have the extension .txt.

For media
Subtitles created for TV broadcast are stored in a variety of file formats. The majority of these formats are proprietary to the vendors of subtitle insertion systems. Broadcast subtitle formats include: .XIF .X32 .PAC .RAC .CHK .AYA .890 .CIP .CAP .ULT .USF .CIN .L32 .ST4 .ST7 .TIT .STL The EBU format defined by Technical Reference 3264-E [14] is an 'open' format intended for subtitle exchange between broadcasters. Files in this format have the extension .stl (not to be mixed up with text "Spruce subtitle format" mentioned above, which also has extension .stl) For internet delivery: SMIL Timed Text DFXP The Timed Text format currently a "Candidate Recommendation" of the W3C (called DFXP[15]) is also proposed as an 'open' format for subtitle exchange and distribution to media players, such as Microsoft Silverlight.