Commit | Line | Data |
---|---|---|
930d75f0 | 1 | Terminal Emulator Multimedia Standard - Proposed Design |
5a16a542 KL |
2 | ======================================================= |
3 | ||
930d75f0 KL |
4 | Version: 1 |
5 | ||
6 | ||
5a16a542 KL |
7 | |
8 | Purpose | |
9 | ------- | |
10 | ||
930d75f0 KL |
11 | Multiple standards exist to incorporate image data in text-based |
12 | terminals and terminal emulators. Few standards have wide adoption | |
13 | despite frequent user requests for these features and hardware support | |
14 | for several of the standards. | |
15 | ||
16 | A group including developers of several widely-used terminal emulators | |
17 | has been working on defining the needs and limitations for a standard | |
18 | that can be implemented in current-gen terminal emulators. The | |
19 | discussion has been primarily captured here: | |
20 | https://gitlab.freedesktop.org/terminal-wg/specifications/issues/12 | |
21 | ||
22 | This document collects many of the reported desires and practical | |
23 | constraints of that discussion into a proposed standard that | |
24 | encompasses three independent new features: | |
25 | ||
26 | 1. A method to transfer multimedia data for immediate display within | |
27 | the screen cell grid ("Direct Multimedia"). | |
28 | ||
29 | 2. A method to transfer multimedia data to a terminal-managed cache, | |
30 | and later display that data within the screen cell grid ("Cached | |
31 | Multimedia"). | |
32 | ||
33 | 3. A method to assign cell data to different layers with options for | |
34 | both layer and cell transparency ("Layers"). | |
35 | ||
36 | A terminal may implement any combination of these features | |
37 | independently of each other. If all features are supported, then all | |
38 | of the design goals outlined in this document can be met. | |
39 | ||
40 | The same mechanisms that can put raster-based images on the screen are | |
41 | also readily generalizable to other media types such as vector-based | |
42 | images and animations. This document is thus a "multimedia" proposal | |
43 | rather than a "simple images" proposal. | |
5a16a542 | 44 | |
5a16a542 KL |
45 | |
46 | ||
47 | Acknowledgements | |
48 | ---------------- | |
49 | ||
50 | This proposal has been informed from the following prior work: | |
51 | ||
52 | * DEC VT300 series sixel graphics standard: | |
53 | https://vt100.net/docs/vt3xx-gp/chapter14.html | |
54 | ||
55 | * iTerm2 image protocol: | |
56 | https://iterm2.com/documentation-images.html | |
57 | ||
58 | * Kitty image protocol: | |
5172fe03 | 59 | https://sw.kovidgoyal.net/kitty/graphics-protocol.html |
5a16a542 KL |
60 | |
61 | * Jexer Terminal User Interface: | |
62 | https://gitlab.com/klamonte/jexer | |
63 | ||
64 | ||
930d75f0 | 65 | |
5a16a542 KL |
66 | Design Goals - Core |
67 | ------------------- | |
68 | ||
69 | The core ("must-have") design goals are: | |
70 | ||
71 | * Be easy to implement in existing terminals and applications: | |
72 | ||
73 | - Sacrifice "10%" of potential function to eliminate "90%" of | |
74 | implementation pain. "Less is more." | |
75 | ||
76 | - Be a strict superset of the existing iTerm2 and DEC sixel image | |
77 | solutions. One should be able to take an existing terminal or | |
78 | application that emits/consumes iTerm2 or sixel sequences, and | |
79 | only change the control sequence introducer/termination to achieve | |
80 | the same effect as a terminal/application that conforms with this | |
81 | standard. | |
82 | ||
83 | * Have no ambiguity. If two terminal or application developers can | |
84 | read this document and reach different conclusions on what should be | |
85 | on the screen, then an error exists in this document that must be | |
930d75f0 | 86 | corrected. |
5a16a542 | 87 | |
895440f3 | 88 | - Every feature must be straightforward to validate via automated |
5a16a542 KL |
89 | unit testing. |
90 | ||
895440f3 KL |
91 | - Every conformant terminal must produce the same output (pixels on |
92 | screen) given the same input (terminal font, terminal sequences). | |
5a16a542 KL |
93 | |
94 | - Every option must have a defined default value. | |
95 | ||
96 | - Erroneous sequences must have defined expected results. | |
97 | ||
98 | - Every operation must act atomically: either everything worked | |
930d75f0 KL |
99 | (image is on screen, cursor has moved, terminal state has changed, |
100 | etc.) or nothing did. | |
101 | ||
102 | * Integrate with existing ECMA-48 / ANSI X3.64 defined sequences: | |
103 | ||
104 | - Operations on Tiles/Cells containing text will have the same | |
105 | effect when applied to Tiles/Cells containing image data. | |
106 | ||
107 | - Existing sequences are given new parameters to cover needed | |
108 | features rather than entirely new sequences introduced. | |
5a16a542 KL |
109 | |
110 | * Be straightforward to implement in non-"physical" terminals, | |
111 | including: | |
112 | ||
113 | - Future versions of terminal control libraries such as ncurses and | |
114 | termbox. | |
115 | ||
116 | - Terminal multiplexers that support "headless" terminals (no | |
117 | physical screen) and "multi-head" terminals (many different | |
118 | physical screens). | |
119 | ||
120 | * Be platform-agnostic, and easy to implement on (at the least): | |
121 | POSIX, Windows, and web. | |
122 | ||
123 | - All features must be available even if the only means of | |
124 | communication between the application and terminal is control | |
125 | sequences (e.g. no shared disk, no shared memory, no shared DOM, | |
126 | etc.). | |
127 | ||
128 | * Support graceful fallback: | |
129 | ||
130 | - Terminal emulators and physical terminals that do not support this | |
131 | standard should remain usable with no undefined screen artifacts, | |
132 | even when the application blindly emits these sequences to those | |
133 | terminals. | |
134 | ||
135 | - This standard must able to be versioned for future enhancements. | |
136 | ||
137 | - An application must be able to detect that its terminal supports | |
138 | this standard, and at what version. | |
139 | ||
140 | * Support secure programming practices: | |
141 | ||
142 | - Applications must not be able to obtain unauthorized data from | |
143 | terminal memory, such as: images emitted by other applications | |
144 | still present in the terminal's scrollback buffer, terminal or | |
145 | system memory limits. | |
146 | ||
147 | - Applications must not be able to compromise the terminal through | |
148 | denial-of-service such as: excessive memory usage, unterminated | |
149 | control sequences. Similarly, terminals must not be able to | |
150 | compromise application through their responses to application | |
151 | queries. | |
152 | ||
153 | - Applications must not be able to manipulate the terminal into | |
154 | performing an insecure operation such as: reading arbitrary shared | |
155 | memory regions, reading arbitrary files on disk, deleting | |
156 | arbitrary files on disk, etc. Similarly, terminals must not be | |
157 | able to manipulate applications into performing insecure | |
158 | operations. | |
159 | ||
160 | - This standard must be implementable when the terminal has a fixed | |
161 | maximum memory, such as a kernel-level device driver. | |
162 | ||
163 | ||
164 | ||
165 | Design Goals - Secondary | |
166 | ------------------------ | |
167 | ||
5172fe03 KL |
168 | The secondary ("nice-to-have") design goals are listed below. These |
169 | might not all be possible, but will kept in mind: | |
5a16a542 KL |
170 | |
171 | * Minimal redundant network traffic for on-screen data that is | |
172 | repeated: either on screen in multiple places, or in the same place | |
173 | but refreshed multiple times. | |
174 | ||
175 | * Asynchronous notification from terminal to application that the | |
176 | screen has been changed by outside or user action. Examples: font | |
177 | change, session detach/attach, user changed image preferences. | |
178 | ||
5172fe03 KL |
179 | * The ability for a multiplexer to "pass-thru" the image drawing |
180 | sequence to its "outer" terminal, with some support for limited | |
181 | clipping. | |
182 | ||
183 | ||
5a16a542 KL |
184 | |
185 | Out Of Scope | |
186 | ------------ | |
187 | ||
930d75f0 | 188 | The following items are out of scope: |
5a16a542 KL |
189 | |
190 | * Bidirectional output. Applications are expected to generate Tiles | |
191 | and place them on screen where they need. The cursor response to | |
930d75f0 KL |
192 | image sequences are defined as left-to-right-top-to-bottom, |
193 | consistent with ECMA-48 / ANSI X3.64 sequences. An independent BIDI | |
194 | standard is free to apply whatever solution will work for ECMA-48 / | |
195 | ANSI X3.64 sequences to the sequences described in this document. | |
5a16a542 | 196 | |
930d75f0 KL |
197 | * Capabilities. This standard defines a limited number of new |
198 | terminal reports and responses. These are not intended to be used | |
199 | as a general-purpose capabilities model. | |
5a16a542 | 200 | |
40e3b486 KL |
201 | * Terminal Cache Management. This standard defines a means for |
202 | applications and terminals to communicate around cached multimedia | |
203 | items, but terminals are free to implement whatever cache management | |
204 | strategies they deem fit. | |
205 | ||
206 | * Reliable Transport. This standard defines a two-way | |
207 | command/response protocol that may get out of order on unreliable | |
208 | channels such as 3-wire RS232. Applictions that require reliable | |
209 | transport on unreliable links may choose to use one of the many | |
210 | successful standards available for this purpose. | |
211 | ||
5a16a542 KL |
212 | |
213 | ||
214 | Definitions | |
215 | ----------- | |
216 | ||
217 | Terminal - The hardware, or a program that simulates hardware, | |
218 | comprising a keyboard, screen, and mouse. | |
219 | ||
220 | Application - A program that utilizes the terminal for its | |
221 | input/output with the user. | |
222 | ||
223 | Multiplexer - A special case of an application that simulates one or | |
224 | more "inner" terminals for other applications to use, | |
225 | and composes these inner terminals into a combined | |
226 | screen to emit to one or more "outer" terminals that | |
227 | obtain input/output from the user. Multiplexers are | |
228 | thus both applications and terminals. | |
229 | ||
5172fe03 KL |
230 | X - The column coordinate of a cell. This standard is 1-based (like |
231 | ECMA-48): the left-most column of the screen is numbered 1. | |
5a16a542 | 232 | |
5172fe03 KL |
233 | Y - The row coordinate of a cell. This standard is 1-based (like |
234 | ECMA-48): the top-most row of the screen is numbered 1. | |
5a16a542 KL |
235 | |
236 | Z - The layer that text or multimedia is placed on. This proposal | |
5172fe03 | 237 | uses a right-hand coordinate system with (X, Y, Z) = (1, 1, 1) |
930d75f0 | 238 | defined as the top-left corner on the default layer; positive Z |
5a16a542 KL |
239 | projects "away" from the user and "into" or "behind" the screen. |
240 | Rendering the Cells on the screen must produce the same result as | |
930d75f0 | 241 | painter's algorithm (see "Layers - Rendering" section below). |
5a16a542 KL |
242 | |
243 | Cell - A fixed-width-and-height rectangle on the screen. The cells of | |
244 | the screen are arranged in a grid of X columns and Y rows. A | |
930d75f0 KL |
245 | Cell has dimensions of cellWidth and cellHeight pixels. Every |
246 | Cell has a coordinate of (X, Y) (or (X, Y, Z) when the terminal | |
247 | supports the layers feature). | |
5a16a542 KL |
248 | |
249 | Tile - One or more contiguous Cells with data to be displayed. The | |
250 | data can be text or image data, but not both. A Tile has width | |
251 | of 1, 2, or more, and a coordinate of (X, Y, Z) that is the | |
252 | same as its left-most (first) Cell's (X, Y, Z). In practice, | |
253 | Tiles are typically one Cell wide for ASCII and Latin language | |
254 | glyphs, and two Cells wide for "fullwidth" glyphs as used in | |
255 | Asian langauges, emojis, and symbols. This standard does not | |
256 | preclude Tiles from encompassing entire grapheme clusters. | |
930d75f0 KL |
257 | Note that ECMA-48 / ANSI X3.64 operations are performed against |
258 | Tiles, not Cells: if a 2-Cell-wide Tile is deleted via | |
259 | backspace, then the cursor will decrement on screen by two | |
260 | columns. | |
5a16a542 KL |
261 | |
262 | Layer - A screen-sized grid of Cells that have the same Z coordinate. | |
263 | Layers are drawn to the screen in descending Z order. Layers | |
264 | may have optional additional attributes such as transparency. | |
930d75f0 KL |
265 | Layer support is an orthogonal (independent) option to |
266 | multimedia support. It is acceptable for terminals to support | |
267 | multimedia without layers and vice versa. | |
268 | ||
269 | ||
270 | ||
930d75f0 KL |
271 | All Features - Detection |
272 | ------------------------ | |
273 | ||
274 | Applications can detect support for these features using Primary | |
275 | Device Attributes (DA) and DECID (ESC Z, or 0x9A). | |
276 | ||
277 | Terminals that support this standard will repond with additional | |
278 | parameter(s): "224" for direct multimedia, "225" for cached | |
279 | multimedia, and "226" for layers. A recap of the parameters xterm | |
280 | supports is listed below, with these new feature responses included: | |
281 | ||
282 | | VT220 (and higher) Response | Description | | |
283 | |-----------------------------|--------------------------------------------| | |
284 | | 1 | 132-columns | | |
285 | | 2 | Printer | | |
286 | | 3 | ReGIS graphics | | |
287 | | 4 | Sixel graphics | | |
288 | | 6 | Selective erase | | |
289 | | 8 | User-defined keys | | |
290 | | 9 | National Replacement Character sets | | |
291 | | 1 5 | Technical characters | | |
292 | | 1 6 | Locator port | | |
293 | | 1 7 | Terminal state interrogation | | |
294 | | 1 8 | User windows | | |
295 | | 2 1 | Horizontal scrolling | | |
296 | | 2 2 | ANSI color, e.g., VT525 | | |
297 | | 2 8 | Rectangular editing | | |
298 | | 2 9 | ANSI text locator (i.e., DEC Locator mode) | | |
299 | | 2 2 4 | Direct Multimedia Version 1 | | |
300 | | 2 2 5 | Cached Multimedia Version 1 | | |
301 | | 2 2 6 | Layers | | |
302 | ||
303 | ||
304 | ||
305 | Direct Multimedia - Summary | |
306 | --------------------------- | |
307 | ||
40e3b486 KL |
308 | Non-text data (multimedia) can be sent to the terminal for immediate |
309 | display in a rectangular (single-layer) region of text Cells. | |
310 | Multimedia data is transmitted to the terminal using one of two wire | |
311 | formats described later in this document. | |
312 | ||
313 | Setting a Cell to multimedia is a destructive operation: the Cell's | |
314 | original text is lost. Multimedia pixels will not overlap rendered | |
315 | text in the same Cell. To achieve pixels overlaid on text, the layers | |
316 | feature can be used. | |
317 | ||
318 | Setting any part of a multi-Cell Tile to multimedia also "breaks up" | |
319 | the Tile into a range of single Cells. In other words, multimedia can | |
320 | only be carried by a Cell, not a Tile. | |
321 | ||
322 | The pixels of a multimedia Cell are assigned to the Cell's foreground; | |
323 | multimedia Cells have no background. If a terminal supports the | |
324 | layers feature, setting a multimedia Cell's foreground transparency to | |
325 | true/enabled causes that Cell to not be displayed at all; setting its | |
326 | background transparency to either true/enabled or false/disabled has | |
327 | no visible effect. | |
328 | ||
329 | The pixels of multimedia Cells can come from two sources: | |
330 | ||
331 | 1. The application can generate pixels and send them to the terminal | |
332 | for display at the current cursor position. | |
333 | ||
334 | 2. The application can specify a source for the multimedia and the | |
335 | terminal will generate the pixels for display at the current | |
336 | cursor position. | |
337 | ||
930d75f0 KL |
338 | |
339 | ||
340 | Direct Multimedia - Required Support For Existing Sequences | |
341 | ----------------------------------------------------------- | |
342 | ||
343 | A terminal with direct multimedia feature must support the following | |
344 | defined xterm sequences: | |
345 | ||
346 | | Sequence | Description | | |
347 | |----------------|-----------------------------------------------------| | |
348 | | CSI 16 t | Responds with CSI 6 ; cellHeight ; cellWidth t | | |
349 | | CSI 18 t | Responds with CSI 8 ; rows ; columns t | | |
350 | ||
351 | ||
352 | ||
353 | Direct Multimedia - New Sequences | |
354 | --------------------------------- | |
355 | ||
40e3b486 KL |
356 | A terminal with direct multimedia feature must support the following |
357 | new sequences: | |
930d75f0 | 358 | |
40e3b486 KL |
359 | | Sequence | Command | Description | |
360 | |--------------------------------------|-------------|-------------------------| | |
361 | | OSC 1 3 3 8 ; s i x e l : {data} BEL | SIXEL | Display sixel at (x, y) | | |
362 | | OSC 1 3 3 8 ; s i x e l : {data} ST | SIXEL | Display sixel at (x, y) | | |
5919b89a KL |
363 | | OSC 1 3 3 8 ; F i l e = {args} : {data} BEL | DMDISPLAY | Display media at (x, y) | |
364 | | OSC 1 3 3 8 ; F i l e = {args} : {data} ST | DMDISPLAY | Display media at (x, y) | | |
365 | | CSI ? 3 0 0 0 h | DECSET 3000 | Enable SCRCHANGE notification | | |
366 | | CSI ? 3 0 0 0 l | DECRST 3000 | Disable SCRCHANGE notification | | |
367 | | OSC 1 3 3 9 ; Pe ; {args} ST | DMRESP | Terminal response to DMDISPLAY | | |
40e3b486 KL |
368 | | CSI ? 3 0 0 1 h | DECSET 3001 | Enable DMDISPLAY responses | |
369 | | CSI ? 3 0 0 1 l | DECRST 3001 | Disable DMDISPLAY responses | | |
5919b89a KL |
370 | |
371 | ||
372 | ||
373 | If SCRCHANGE is set/enabled, then the terminal will send the "CSI 6 ; | |
374 | cellHeight ; cellWidth t" when the font size has changed, and "CSI 8 ; | |
375 | rows ; columns t" when the number of rows/columns on the screen has | |
376 | changed. | |
377 | ||
40e3b486 KL |
378 | |
379 | ||
380 | For the SIXEL command: | |
381 | ||
382 | * The {data} is a sixel sequence as described in the VT330/340 | |
383 | Programmer Reference Manual, Chapter 14, available online at: | |
384 | http://vt100.net/docs/vt3xx-gp/chapter14.html . The {data} is the | |
385 | "P1 ; P2 ; P3 ; q s..s" portion of the Device Control String, i.e. a | |
386 | complete sixel sequence minus the leading DCS and trailing ST. | |
387 | ||
388 | * The sixel image is processed as shown below. Note that this | |
389 | behavior is equivalent to Sixel Scrolling mode enabled. | |
390 | ||
391 | - The sixel active position starts at the upper-left corner of the | |
392 | text cursor position. | |
393 | ||
394 | - The screen is scrolled up if the image overflows into the bottom | |
395 | text row. | |
396 | ||
397 | - Pixels that would be drawn to the right of the visible region on | |
398 | screen are discarded. | |
399 | ||
400 | - The cursor's final position is on the same column as the starting | |
401 | cursor position, and on the row immediately below the image. | |
402 | ||
403 | ||
404 | For the DMDISPLAY command: | |
405 | ||
5919b89a | 406 | * The {args} is a set of key-value pairs (each pair separated by |
40e3b486 | 407 | semicolon (';')), followed by a colon (':'), followed by a base-64 |
5919b89a | 408 | encoded string ({data}). |
40e3b486 KL |
409 | |
410 | * A key can be any alpha-numeric ASCII string ('0' - '9', 'A' - 'Z', | |
411 | 'a' - 'z'). | |
412 | ||
413 | * A value is any printable ASCII string not containing whitespace, | |
414 | colon, or semicolon ('!' - '9', '<' - '~'). | |
415 | ||
416 | * Any alpha-numeric key may be specified. A key that is not supported | |
417 | by the terminal is ignored without error. | |
418 | ||
419 | * The multimedia pixels are processed as shown below. | |
420 | ||
421 | - The pixel are drawn starting at the upper-left corner of the text | |
422 | cursor position. | |
423 | ||
424 | - If scrolling is specified as enabled, then: | |
425 | ||
426 | a. The screen is scrolled up if the image overflows into the | |
427 | bottom text row. | |
428 | ||
429 | b. The cursor's final position is on the same column as the | |
430 | starting cursor position, and on the row immediately below the | |
431 | image. | |
432 | ||
433 | - If scrolling is omitted or specified as disabled, then: | |
434 | ||
435 | a. The screen is never scrolled. | |
436 | ||
437 | b. Pixels that would be drawn below the visible region on screen | |
438 | are discarded. | |
439 | ||
440 | c. The cursor's final position is at the same column and row as | |
441 | the starting cursor position, i.e. the cursor does not move at | |
442 | all. | |
443 | ||
444 | - Pixels that would be drawn to the right of the visible region on | |
445 | screen are discarded. | |
446 | ||
447 | ||
448 | ||
449 | The keys for the key-value pairs that must be supported by the | |
450 | terminal are listed below: | |
451 | ||
452 | | Key | Default Value | Description | | |
453 | |--------------|---------------|----------------------------------------------| | |
5919b89a KL |
454 | | type | "image/rgb" | mime-type describing data field | |
455 | | url | "" | If set, a location containing the media data | | |
40e3b486 KL |
456 | | width | 1 | Number of Cells or pixels wide to display in | |
457 | | height | 1 | Number of Cells or pixels high to display in | | |
458 | | scale | "none" | Scale/zoom option, see below | | |
459 | | align | "nw" | Align image to edge option, see below | | |
5919b89a KL |
460 | | sourceX | 0 | Media source X position to display | |
461 | | sourceY | 0 | Media source Y position to display | | |
462 | | sourceWidth | "auto" | Media width in pixels to display | | |
463 | | sourceHeight | "auto" | Media height in pixels to display | | |
40e3b486 KL |
464 | |
465 | A terminal may support additional keys. If a key is specified but not | |
466 | supported by the terminal, then it is ignored without error. | |
930d75f0 | 467 | |
930d75f0 KL |
468 | |
469 | ||
5919b89a KL |
470 | The "type" value is a mime-type string describing the format of the |
471 | base64-encoded binary data. The terminal must support at mimunum these | |
472 | mime-types: | |
473 | ||
474 | | Type String | Description | | |
475 | |---------------|--------------------------------------------------------------| | |
476 | | "image/rgb" | Big-endian-encoded 24-bit red, green, blue values | | |
477 | | "image/rgba" | Big-endian-encoded 32-bit red, green, blue, alpha values | | |
478 | | "image/png" | PNG file data as described by (reference to PNG format) | | |
479 | ||
480 | A terminal may support additional types. An application can detect | |
481 | terminal support for a format by: enabling terminal responses (DECSET | |
482 | 3001), sending a DMDISPLAY command, and examining the terminal's | |
483 | response sequence for success or error. | |
484 | ||
485 | ||
486 | ||
487 | The "url" value is a RFC-XXXX defined Universal Resource Located, | |
488 | encoded in RFC-XXXX form as a printable ASCII string not containing: | |
489 | whitespace, colon (':'), semicolon (';'), or equals ('='). | |
490 | ||
491 | A terminal is not required to support any URLs. | |
492 | ||
493 | ||
494 | ||
40e3b486 | 495 | The "width" and "height" values can take the following forms: |
930d75f0 | 496 | |
40e3b486 KL |
497 | | Value | Meaning | |
498 | |-------------------------------|---------------------------| | |
499 | | N (a positive integer) | Number of Cells | | |
500 | | Npx (positive integer + "px") | Number of pixels | | |
501 | | N% (positive integer + "%") | Percent of screen width or height | | |
502 | | "auto" | Number of pixels as defined by the multimedia data | | |
930d75f0 KL |
503 | |
504 | ||
505 | ||
40e3b486 KL |
506 | The "scale" value can take the following values: |
507 | ||
508 | | Value | Meaning | | |
509 | |------------|---------------------------------------------------------------| | |
510 | | "none" | No scaling along either axis. | | |
511 | | "scale" | Stretch image, preserving aspect ratio, to maximum size in the target area without cropping | | |
512 | | "stretch" | Stretch along both axes, distorting aspect ratio, to fill the target area | | |
513 | | "crop" | Stretch along both axes, preserving aspect ration, to completely fill the target area, cropping pixels that will not fit | | |
514 | ||
515 | ||
516 | ||
517 | The "align" value can take the following values: | |
518 | ||
519 | | Value | Meaning | | |
520 | |------------|-----------------------------------------------------------------| | |
521 | | "nw" | Media is placed at the top-left corner (northwest) | | |
522 | | "n" | Media is placed on the top and centered horizontally (north) | | |
523 | | "ne" | Media is placed at the top-right corner (northest) | | |
524 | | "w" | Media is placed on the left and centered vertically (west) | | |
525 | | "c" | Media is centered in the target area (center) | | |
526 | | "e" | Media is placed on the right and centered vertically (east) | | |
527 | | "sw" | Media is placed on the bottom-left corner (southwest) | | |
528 | | "s" | Media is placed on the bottom and centered horizontally (south) | | |
529 | | "se" | Media is placed on the bottom-right corner (southeast) | | |
530 | ||
531 | ||
532 | ||
5919b89a KL |
533 | "sourceX", "sourceY", "sourceWidth", and "sourceHeight" define the |
534 | rectangle of pixels from the media that will be displayed on the | |
535 | screen. The ranges for these values is shown below: | |
40e3b486 | 536 | |
5919b89a KL |
537 | | Key | Minimum Value | Maximum Value | Default Value | |
538 | |--------------|---------------|-------------------------------|---------------| | |
539 | | sourceX | 0 | Media's full width - 1 | 0 | | |
540 | | sourceY | 0 | Media's full height - 1 | 0 | | |
541 | | sourceWidth | 1 | Media's full width - sourceX | "auto" | | |
542 | | sourceHeight | 1 | Media's full height - sourceY | "auto" | | |
40e3b486 | 543 | |
5919b89a KL |
544 | If any of these values are specified and outside the range, no image |
545 | is displayed, and the cursor does not move. "sourceWidth" and | |
546 | "sourceHeight" can be "auto", which means use the maximum available | |
547 | width/height (given sourceX/sourceY) from the media's inherent | |
548 | dimensions. | |
40e3b486 KL |
549 | |
550 | ||
551 | ||
552 | Direct Multimedia - Terminal Responses / Error Handling | |
553 | ------------------------------------------------------- | |
554 | ||
555 | If DMDISPLAY reponses are enabled, then a terminal will respond to the | |
556 | DMDISPLAY display with DMRESP. DMRESP responses must be sent in the | |
557 | same sequential order as the DMDISPLAY commands they are responses to: | |
558 | the terminal may not re-order responses. | |
559 | ||
560 | No provision is made for reliable delivery. On unreliable links | |
561 | (example: 3-wire RS232), the DMDISPLAY and DMRESP command/response | |
562 | sequence may get out of order. | |
563 | ||
564 | ||
565 | ||
566 | The format of DMRESP is: | |
567 | ||
568 | * Pe - a non-negative integer error code. | |
569 | ||
5919b89a | 570 | * The {args} is a set of key-value pairs (each pair separated by |
40e3b486 KL |
571 | semicolon (';')). |
572 | ||
573 | * A key can be any alpha-numeric ASCII string ('0' - '9', 'A' - 'Z', | |
574 | 'a' - 'z'). | |
575 | ||
576 | * A value is any printable ASCII string not containing whitespace, | |
577 | colon, or semicolon ('!' - '9', '<' - '~'). | |
578 | ||
579 | ||
5919b89a | 580 | |
40e3b486 KL |
581 | The Pe error codes are defined as: |
582 | ||
5919b89a | 583 | | Value | Meaning | {args} containts | |
40e3b486 KL |
584 | |-------|------------------------------------|--------------------------| |
585 | | 0 | No error occurred, i.e. success | nothing | | |
5919b89a | 586 | | 1 | Unsupported "type" | "type" value that was incorrect | |
40e3b486 KL |
587 | | 2 | Invalid value - no media displayed | "key" that was incorrect | |
588 | | 3 | Unsupported key - media displayed | "key" that unsupported | | |
589 | | 4 | Insufficient memory | nothing | | |
590 | | 5 | Other error - no media displayed | nothing | | |
591 | | 6 | Other - media displayed | nothing | | |
592 | | 7 | Conflicting keys - no media displayed | nothing | | |
5919b89a | 593 | | 8 | RESERVED FOR FUTURE USE | RESERVED FOR FUTURE USE | |
40e3b486 KL |
594 | |
595 | Additional Pe error codes may be returned; any Pe value except 0, 3, | |
596 | and 6 must mean that the media was not displayed, and the cursor was | |
597 | not moved. | |
598 | ||
599 | If both "type" and "url" are set, no media is diaplyed, the cursor is | |
600 | not moved, and the DMRESP error code is 7. | |
930d75f0 KL |
601 | |
602 | ||
603 | ||
604 | Direct Multimedia - Examples | |
605 | ---------------------------- | |
606 | ||
607 | ||
608 | ||
609 | Cached Multimedia - Summary | |
610 | --------------------------- | |
611 | ||
40e3b486 KL |
612 | Non-text data (multimedia) can be sent to the terminal for later |
613 | display in a rectangular (single-layer) region of text Cells. | |
614 | Multimedia data is transmitted to the terminal using the CMCACHE | |
615 | command described below, and displayed on screen using the CMDISPLAY | |
616 | command. A single CMCACHE command can support many CMDISPLAY | |
617 | commands. | |
618 | ||
619 | Upon display, setting a Cell to multimedia is a destructive operation: | |
620 | the Cell's original text is lost. Multimedia pixels will not overlap | |
621 | rendered text in the same Cell. To achieve pixels overlaid on text, | |
622 | the layers feature can be used. | |
623 | ||
624 | Setting any part of a multi-Cell Tile to multimedia also "breaks up" | |
625 | the Tile into a range of single Cells. In other words, multimedia can | |
626 | only be carried by a Cell, not a Tile. | |
627 | ||
628 | The pixels of a multimedia Cell are assigned to the Cell's foreground; | |
629 | multimedia Cells have no background. If a terminal supports the | |
630 | layers feature, setting a multimedia Cell's foreground transparency to | |
631 | true/enabled causes that Cell to not be displayed at all; setting its | |
632 | background transparency to either true/enabled or false/disabled has | |
633 | no visible effect. | |
634 | ||
635 | The pixels of multimedia Cells can come from two sources: | |
636 | ||
637 | 1. The application can generate pixels and send them to the terminal | |
638 | for display at the current cursor position. | |
639 | ||
640 | 2. The application can specify a source for the multimedia and the | |
641 | terminal will generate the pixels for display at the current | |
642 | cursor position. | |
643 | ||
644 | ||
645 | ||
646 | ||
647 | Cached Multimedia - Cache/Memory Management | |
648 | ------------------------------------------- | |
930d75f0 | 649 | |
5919b89a KL |
650 | The terminal manages a cache of multimedia data on behalf of one or |
651 | more applications. Applications request media be stored in the cache, | |
652 | and if successful the terminal provides an identification number that | |
653 | applications must use to request display from the cache to the screen. | |
930d75f0 | 654 | |
5919b89a KL |
655 | The amount of memory and retention/eviction strategy for the cache is |
656 | wholly managed by the terminal, with the following restrictions: | |
930d75f0 | 657 | |
5919b89a KL |
658 | * The terminal may not remove items from the cache that have any |
659 | portion being actively displayed on the primary or alternate | |
660 | screens. | |
40e3b486 | 661 | |
5919b89a KL |
662 | * The terminal must respond to every CMCACHE command with a new unique |
663 | ID. | |
40e3b486 | 664 | |
5919b89a KL |
665 | The scrollback buffer is permitted, and recommended, to contain only a |
666 | few (or zero) multimedia images. Terminals should consider retaining | |
667 | only the last 2-5 screens' worth of pixel data in the scrollback | |
668 | buffer. | |
930d75f0 KL |
669 | |
670 | ||
671 | ||
672 | Cached Multimedia - Required Support For Existing Sequences | |
673 | ----------------------------------------------------------- | |
674 | ||
675 | A terminal with cached multimedia feature must support the following | |
676 | defined xterm sequences: | |
677 | ||
678 | | Sequence | Description | | |
679 | |----------------|-----------------------------------------------------| | |
680 | | CSI 16 t | Responds with CSI 6 ; cellHeight ; cellWidth t | | |
681 | | CSI 18 t | Responds with CSI 8 ; rows ; columns t | | |
682 | ||
683 | ||
684 | ||
685 | Cached Multimedia - New Sequences | |
686 | --------------------------------- | |
687 | ||
40e3b486 KL |
688 | A terminal with cached multimedia feature must support the following new |
689 | sequences: | |
690 | ||
691 | | Sequence | Command | Description | | |
692 | |--------------------------------------|-----------|-------------------------| | |
5919b89a KL |
693 | | CSI ? 3 0 0 0 h | DECSET 3000 | Enable SCRCHANGE notification | |
694 | | CSI ? 3 0 0 0 l | DECRST 3000 | Disable SCRCHANGE notification | | |
695 | | OSC 1 3 4 0 ; F i l e = {args} : {data} BEL | CMCACHE | Display media at (x, y) | | |
696 | | OSC 1 3 4 1 ; Pi ; {args} ST | CMDISPLAY | Display media at (x, y) | | |
697 | | OSC 1 3 4 2 ; Pi ; Pe ; {args} ST | CMCRESP | Terminal response to CMCACHE | | |
698 | | OSC 1 3 4 3 ; Pi ; Pe ; {args} ST | CMDRESP | Terminal response to CMDISPLAY | | |
40e3b486 KL |
699 | |
700 | ||
701 | ||
5919b89a KL |
702 | If SCRCHANGE is set/enabled, then the terminal will send the "CSI 6 ; |
703 | cellHeight ; cellWidth t" when the font size has changed, and "CSI 8 ; | |
704 | rows ; columns t" when the number of rows/columns on the screen | |
705 | changes. | |
706 | ||
707 | ||
40e3b486 KL |
708 | |
709 | Cached Multimedia - CMCACHE | |
710 | --------------------------- | |
711 | ||
712 | For the CMCACHE command: | |
713 | ||
5919b89a KL |
714 | * The {args} is a set of key-value pairs (each pair separated by |
715 | semicolon (';')), followed by a colon (':'), followed by a base-64 | |
716 | encoded string ({data}). | |
40e3b486 KL |
717 | |
718 | * A key can be any alpha-numeric ASCII string ('0' - '9', 'A' - 'Z', | |
719 | 'a' - 'z'). | |
720 | ||
721 | * A value is any printable ASCII string not containing whitespace, | |
722 | colon, or semicolon ('!' - '9', '<' - '~'). | |
723 | ||
5919b89a KL |
724 | |
725 | ||
40e3b486 KL |
726 | The keys for the key-value pairs that must be supported by the |
727 | terminal are listed below: | |
728 | ||
729 | | Key | Default Value | Description | | |
730 | |--------------|---------------|----------------------------------------------| | |
731 | | type | "image/rgb" | mime-type describing data field | | |
732 | | url | "" | If set, a location containing the media data | | |
733 | ||
734 | ||
5919b89a | 735 | |
40e3b486 KL |
736 | The "type" value is a mime-type string describing the format of the |
737 | base64-encoded binary data. The terminal must support at mimunum these | |
738 | mime-types: | |
739 | ||
740 | | Type String | Description | | |
741 | |---------------|--------------------------------------------------------------| | |
742 | | "image/rgb" | Big-endian-encoded 24-bit red, green, blue values | | |
743 | | "image/rgba" | Big-endian-encoded 32-bit red, green, blue, alpha values | | |
744 | | "image/png" | PNG file data as described by (reference to PNG format) | | |
745 | ||
746 | A terminal may support additional types. An application can detect | |
747 | terminal support for a format by: sending a CMCACHE command, and | |
748 | examining the terminal's CMCRESP sequence for success or error. | |
749 | ||
750 | ||
751 | ||
5919b89a KL |
752 | The "url" value is a RFC-XXXX defined Universal Resource Located, |
753 | encoded in RFC-XXXX form as a printable ASCII string not containing: | |
754 | whitespace, colon (':'), semicolon (';'), or equals ('='). | |
755 | ||
756 | A terminal is not required to support any URLs. | |
757 | ||
758 | ||
759 | ||
40e3b486 KL |
760 | Cached Multimedia - CMDISPLAY |
761 | ----------------------------- | |
762 | ||
763 | For the CMDISPLAY command: | |
764 | ||
765 | * Pi - a non-negative integer media ID that was returned by a CMCRESP | |
766 | response to a previous CMCACHE command. | |
767 | ||
5919b89a | 768 | * The {args} is a set of key-value pairs (each pair separated by |
40e3b486 KL |
769 | semicolon (';')), followed by a colon (':'), followed by a base-64 |
770 | encoded string. | |
771 | ||
772 | * A key can be any alpha-numeric ASCII string ('0' - '9', 'A' - 'Z', | |
773 | 'a' - 'z'). | |
774 | ||
775 | * A value is any printable ASCII string not containing whitespace, | |
776 | colon, or semicolon ('!' - '9', '<' - '~'). | |
777 | ||
778 | * Any alpha-numeric key may be specified. A key that is not supported | |
779 | by the terminal is ignored without error. | |
780 | ||
781 | * The multimedia pixels are processed as shown below. | |
782 | ||
783 | - The pixel are drawn starting at the upper-left corner of the text | |
784 | cursor position. | |
785 | ||
786 | - The screen is never scrolled. | |
787 | ||
788 | - The cursor's final position is at the same column and row as the | |
789 | starting cursor position, i.e. the cursor does not move at all. | |
790 | ||
791 | - Pixels that would be drawn below the visible region on screen are | |
792 | discarded. | |
793 | ||
794 | - Pixels that would be drawn to the right of the visible region on | |
795 | screen are discarded. | |
796 | ||
797 | ||
798 | ||
799 | The keys for the key-value pairs that must be supported by the | |
800 | terminal are listed below: | |
801 | ||
802 | | Key | Default Value | Description | | |
803 | |--------------|---------------|----------------------------------------------| | |
804 | | width | 1 | number of Cells or pixels wide to display in | | |
805 | | height | 1 | number of Cells or pixels high to display in | | |
806 | | scale | "none" | scale/zoom option, see below | | |
807 | | align | "nw" | align image to edge option, see below | | |
5919b89a KL |
808 | | sourceX | 0 | Media source X position to display | |
809 | | sourceY | 0 | Media source Y position to display | | |
810 | | sourceWidth | "auto" | Media width in pixels to display | | |
811 | | sourceHeight | "auto" | Media height in pixels to display | | |
40e3b486 KL |
812 | |
813 | A terminal may support additional keys. If a key is specified but not | |
814 | supported by the terminal, then it is ignored without error. | |
815 | ||
816 | ||
817 | ||
818 | The "width" and "height" values can take the following forms: | |
819 | ||
820 | | Value | Meaning | | |
821 | |-------------------------------|---------------------------| | |
822 | | N (a positive integer) | Number of Cells | | |
823 | | Npx (positive integer + "px") | Number of pixels | | |
824 | | N% (positive integer + "%") | Percent of screen width or height | | |
825 | | "auto" | Number of pixels as defined by the multimedia data | | |
826 | ||
827 | ||
828 | ||
829 | The "scale" value can take the following values: | |
830 | ||
831 | | Value | Meaning | | |
832 | |------------|---------------------------------------------------------------| | |
833 | | "none" | No scaling along either axis. | | |
834 | | "scale" | Stretch image, preserving aspect ratio, to maximum size in the target area without cropping | | |
835 | | "stretch" | Stretch along both axes, distorting aspect ratio, to fill the target area | | |
836 | | "crop" | Stretch along both axes, preserving aspect ration, to completely fill the target area, cropping pixels that will not fit | | |
837 | ||
838 | ||
839 | ||
840 | The "align" value can take the following values: | |
841 | ||
842 | | Value | Meaning | | |
843 | |------------|-----------------------------------------------------------------| | |
844 | | "nw" | Media is placed at the top-left corner (northwest) | | |
845 | | "n" | Media is placed on the top and centered horizontally (north) | | |
846 | | "ne" | Media is placed at the top-right corner (northest) | | |
847 | | "w" | Media is placed on the left and centered vertically (west) | | |
848 | | "c" | Media is centered in the target area (center) | | |
849 | | "e" | Media is placed on the right and centered vertically (east) | | |
850 | | "sw" | Media is placed on the bottom-left corner (southwest) | | |
851 | | "s" | Media is placed on the bottom and centered horizontally (south) | | |
852 | | "se" | Media is placed on the bottom-right corner (southeast) | | |
853 | ||
930d75f0 KL |
854 | |
855 | ||
5919b89a KL |
856 | "sourceX", "sourceY", "sourceWidth", and "sourceHeight" define the |
857 | rectangle of pixels from the media that will be displayed on the | |
858 | screen. The ranges for these values is shown below: | |
859 | ||
860 | | Key | Minimum Value | Maximum Value | Default Value | | |
861 | |--------------|---------------|-------------------------------|---------------| | |
862 | | sourceX | 0 | Media's full width - 1 | 0 | | |
863 | | sourceY | 0 | Media's full height - 1 | 0 | | |
864 | | sourceWidth | 1 | Media's full width - sourceX | "auto" | | |
865 | | sourceHeight | 1 | Media's full height - sourceY | "auto" | | |
866 | ||
867 | If any of these values are specified and outside the range, no image | |
868 | is displayed, and the cursor does not move. "sourceWidth" and | |
869 | "sourceHeight" can be "auto", which means use the maximum available | |
870 | width/height (given sourceX/sourceY) from the media's inherent | |
871 | dimensions. | |
872 | ||
873 | ||
874 | ||
930d75f0 KL |
875 | Cached Multimedia - Error Handling |
876 | ---------------------------------- | |
877 | ||
40e3b486 KL |
878 | A terminal will always respond to the CMCACHE command with CMCRESP, |
879 | and to the CMDISPLAY command with CMDRESP. Responses must be sent in | |
880 | the same sequential order as the CMCACHE/CMDISPLAY commands they are | |
881 | responses to: the terminal may not re-order responses. | |
930d75f0 | 882 | |
40e3b486 KL |
883 | No provision is made for reliable delivery. On unreliable links |
884 | (example: 3-wire RS232), the command/response sequence may get out of | |
885 | order. | |
930d75f0 | 886 | |
930d75f0 KL |
887 | |
888 | ||
40e3b486 KL |
889 | Cached Multimedia - Error Handling - CMCRESP |
890 | -------------------------------------------- | |
930d75f0 | 891 | |
40e3b486 KL |
892 | The format of CMCRESP is: |
893 | ||
894 | * Pi - a non-negative integer media ID. The terminal will generate a | |
895 | new ID for every image successfully loaded into the cache. The | |
896 | application must use this ID for CMDISPLAY commands. | |
897 | ||
898 | * Pe - a non-negative integer error code. | |
899 | ||
5919b89a | 900 | * The {args} is a set of key-value pairs (each pair separated by |
40e3b486 KL |
901 | semicolon (';')). |
902 | ||
903 | * A key can be any alpha-numeric ASCII string ('0' - '9', 'A' - 'Z', | |
904 | 'a' - 'z'). | |
930d75f0 | 905 | |
40e3b486 KL |
906 | * A value is any printable ASCII string not containing whitespace, |
907 | colon, or semicolon ('!' - '9', '<' - '~'). | |
5a16a542 KL |
908 | |
909 | ||
5919b89a | 910 | |
40e3b486 KL |
911 | The Pe error codes are defined as: |
912 | ||
5919b89a KL |
913 | | Value | Meaning | {args} containts | |
914 | |-------|----------------------------------------|--------------------------| | |
915 | | 0 | No error occurred, i.e. success | nothing | | |
916 | | 1 | Unsupported "type" | "type" value that was incorrect | | |
917 | | 2 | Invalid value - no media stored | "key" that was incorrect | | |
918 | | 3 | Unsupported key - media stored | "key" that unsupported | | |
919 | | 4 | Insufficient memory - no media stored | nothing | | |
920 | | 5 | Other error - no media stored | nothing | | |
921 | | 6 | Other - media stored | nothing | | |
922 | | 7 | Conflicting keys - no media stored | nothing | | |
923 | | 8 | RESERVED FOR FUTURE USE | RESERVED FOR FUTURE USE | | |
40e3b486 KL |
924 | |
925 | Additional Pe error codes may be returned; any Pe value except 0, 3, | |
926 | and 6 must mean that the media was not stored in the cache. | |
927 | ||
928 | If both "type" and "url" are set, no media is diaplyed, the cursor is | |
929 | not moved, and the CMCRESP error code is 7. | |
930 | ||
931 | ||
932 | ||
933 | Cached Multimedia - Error Handling - CMDRESP | |
934 | -------------------------------------------- | |
935 | ||
936 | The format of CMDRESP is: | |
937 | ||
938 | * Pi - a non-negative integer media ID. | |
939 | ||
940 | * Pe - a non-negative integer error code. | |
941 | ||
5919b89a | 942 | * The {args} is a set of key-value pairs (each pair separated by |
40e3b486 KL |
943 | semicolon (';')). |
944 | ||
945 | * A key can be any alpha-numeric ASCII string ('0' - '9', 'A' - 'Z', | |
946 | 'a' - 'z'). | |
947 | ||
948 | * A value is any printable ASCII string not containing whitespace, | |
949 | colon, or semicolon ('!' - '9', '<' - '~'). | |
950 | ||
951 | ||
5919b89a | 952 | |
40e3b486 KL |
953 | The Pe error codes are defined as: |
954 | ||
5919b89a KL |
955 | | Value | Meaning | {args} containts | |
956 | |-------|----------------------------------------|--------------------------| | |
957 | | 0 | No error occurred, i.e. success | nothing | | |
958 | | 1 | RESERVED FOR FUTURE USE | RESERVED FOR FUTURE USE | | |
40e3b486 KL |
959 | | 2 | Invalid value - no media displayed | "key" that was incorrect | |
960 | | 3 | Unsupported key - media displayed | "key" that unsupported | | |
961 | | 4 | Insufficient memory - no media displayed | nothing | | |
962 | | 5 | Other error - no media displayed | nothing | | |
963 | | 6 | Other - media displayed | nothing | | |
5919b89a KL |
964 | | 7 | RESERVED FOR FUTURE USE | RESERVED FOR FUTURE USE | |
965 | | 8 | Media was evicted - no media displayed | nothing | | |
40e3b486 KL |
966 | |
967 | Additional Pe error codes may be returned; any Pe value except 0, 3, | |
968 | and 6 must mean that the media was not displayed. | |
5a16a542 | 969 | |
930d75f0 KL |
970 | |
971 | ||
972 | Cached Multimedia - Examples | |
973 | ---------------------------- | |
974 | ||
975 | ||
976 | ||
977 | ||
978 | Layers - Summary | |
979 | ---------------- | |
980 | ||
981 | Layers introduce the concept of a layer "Z" coordinate to the existing | |
982 | rows ("Y") by columns ("X") grid. Put another way, the | |
983 | two-dimensional grid of columns-by-rows becomes a three-dimensional | |
984 | cube of columns-by-rows-by-layers. For this document, the column, | |
985 | row, and layer coordinates are referred to as X, Y, and Z. This | |
986 | cartesian coordinate system is right-handed, with the Z axis pointing | |
987 | "away" from the user "into" the screen. | |
988 | ||
989 | An application treats the Z coordinate exactly as it does X and Y | |
990 | (rows and columns) coordinates: | |
991 | ||
992 | * If it attemps to set Z to a value less than 1, then Z is set to 1. | |
993 | ||
994 | * If it attempts to set Z to a value greater than the number of | |
995 | layers, then Z is set to the number of layers. | |
996 | ||
40e3b486 KL |
997 | New sequences are provided to set and query Z, Y, X; to set and query |
998 | the screen cube size; and control visibility of Cells in-front-of | |
930d75f0 KL |
999 | other Cells. |
1000 | ||
40e3b486 KL |
1001 | Operations that can act on more than one Cell are defined such to act |
1002 | on all layers simultaneously by default; most of these operations can | |
1003 | also be set to act only on the current layer. | |
930d75f0 KL |
1004 | |
1005 | ||
1006 | ||
1007 | Layers - Number of Layers | |
1008 | ------------------------- | |
1009 | ||
1010 | A terminal is required to provide between 1 and a finite number of | |
1011 | layers. | |
1012 | ||
1013 | The number of layers may be different between the primary and | |
1014 | alternate screens. | |
1015 | ||
1016 | An application may request that the terminal allocate additional | |
1017 | layers. The terminal is free to honor or ignore such requests as it | |
1018 | sees fit. | |
1019 | ||
1020 | The scrollback buffer is permitted, and recommended, to contain only a | |
1021 | "flattened" single layer. | |
1022 | ||
1023 | ||
1024 | ||
1025 | Layers - Terminal State | |
1026 | ----------------------- | |
1027 | ||
1028 | The terminal maintains a complex state at all times. This state | |
1029 | includes variables such as cursor position, foreground/background | |
1030 | color, attributes to apply to the next displayed character, and so on. | |
1031 | The layers feature adds more variables to the state, and these | |
1032 | variables are required to be stored with DECSC (ESC 7) and restored | |
1033 | with DECRC (ESC 8). The new variables are listed below: | |
1034 | ||
203cfc23 KL |
1035 | | Mnemonic | Description | Default value | |
1036 | |----------|-----------------------------|----------------| | |
1037 | | Z | Cursor position Z | 1 | | |
1038 | | MSL | Manipulate single layer | off / disabled | | |
1039 | | TFT | Text foreground transparent | false | | |
1040 | | TBT | Text background transparent | false | | |
930d75f0 KL |
1041 | |
1042 | ||
1043 | ||
1044 | Layers - Required Support For Existing Sequences | |
1045 | ------------------------------------------------ | |
1046 | ||
1047 | A terminal with layers feature must support the standard VT100/VT102 | |
1048 | sequences defined in their respective manuals. | |
1049 | ||
1050 | ||
1051 | ||
1052 | Layers - New Sequences | |
1053 | ---------------------- | |
1054 | ||
1055 | A terminal with layer feature must support the following new | |
1056 | sequences: | |
1057 | ||
1058 | | Sequence | Command | Description | | |
1059 | |-------------------|-------------|----------------------------------------| | |
1060 | | CSI ? z ; y ; x H | CUPZ | Move cursor to (x, y, z) | | |
40e3b486 KL |
1061 | | CSI 2 2 5 ; 1 ; Pa t | SLA | Set layer alpha | |
1062 | | CSI ? 3 0 0 2 h | DECSET 3002 | Enable Manupulate Single Layer (MSL) | | |
1063 | | CSI ? 3 0 0 2 l | DECRST 3002 | Disable Manupulate Single Layer (MSL) | | |
930d75f0 KL |
1064 | | CSI ? l ; h ; w t | RSZCUBE | Resize cube to (layers, height, width) | |
1065 | ||
1066 | Default parameters and ranges are listed below: | |
1067 | ||
1068 | | Command | Position / Variable | Default Value | Minumum | Maximum | | |
1069 | |---------|---------------------|---------------|---------|-----------| | |
1070 | | CUPZ | 1 / z | 1 | 1 | # layers | | |
1071 | | CUPZ | 2 / y | 1 | 1 | # rows | | |
1072 | | CUPZ | 3 / x | 1 | 1 | # columns | | |
1073 | | SLA | 1 / alpha | 255 | 0 | 255 | | |
1074 | | RSZCUBE | 1 / l | 1 | 1 | varies | | |
40e3b486 KL |
1075 | | RSZCUBE | 2 / h | 24 | 1 | varies | |
1076 | | RSZCUBE | 3 / w | 80 | 1 | varies | | |
930d75f0 KL |
1077 | |
1078 | The terminal must also support the following new queries: | |
1079 | ||
191ed82c | 1080 | | Query | Response | Description | |
02c14905 | 1081 | |-----------------|-----------------------|--------------------------------| |
191ed82c | 1082 | | CSI ? 1 0 0 n | CSI ? z ; y ; x n | Report cursor Z, Y, X position | |
930d75f0 KL |
1083 | | CSI ? 1 8 t | CSI ? 8 ; l ; h ; w t | Report the text area cube layers, height, width | |
1084 | ||
930d75f0 KL |
1085 | The terminal must support the following new Set Graphics Rendition |
1086 | (SGR) character attributes commands: | |
1087 | ||
1088 | | SGR Parameter | Description | | |
1089 | |---------------|---------------------------------------------| | |
40e3b486 KL |
1090 | | 2 3 0 | Set text foreground color to transparent | |
1091 | | 2 3 9 | Set text foreground color to solid (opaque) | | |
1092 | | 2 4 0 | Set text background color to transparent | | |
1093 | | 2 4 9 | Set text background color to solid (opaque) | | |
930d75f0 KL |
1094 | |
1095 | ||
1096 | ||
1097 | Layers - Error Handling | |
1098 | ----------------------- | |
1099 | ||
1100 | No additional error reporting is provided for layer feature. | |
1101 | ||
1102 | ||
1103 | ||
1104 | Layers - Rendering | |
1105 | ------------------ | |
1106 | ||
1107 | A terminal with layer feature will display its Cells such that the | |
1108 | screen will appear as if it was rendered in the manner of the | |
1109 | pseudo-code below: | |
5a16a542 KL |
1110 | |
1111 | ``` | |
1112 | for each layer Z, in descending order from maxZ to minZ: | |
930d75f0 | 1113 | |
5a16a542 | 1114 | for each row Y, in ascending order from minY to maxY: |
930d75f0 | 1115 | |
5a16a542 | 1116 | for each column X, in ascending order from minX to maxX: |
930d75f0 KL |
1117 | |
1118 | if tile at (X, Y, Z) background color is solid: | |
1119 | draw rectangle of background color with layer alpha | |
1120 | ||
1121 | if tile at (X, Y, Z) foreground color is solid: | |
1122 | if tile at (X, Y, Z) is glyph: | |
1123 | draw glyph with foreground color with layer alpha | |
1124 | else | |
1125 | draw pixel data of tile as red/green/blue/alpha pixels with | |
1126 | layer alpha | |
1127 | ||
5a16a542 KL |
1128 | advance X by tile width |
1129 | next column | |
930d75f0 | 1130 | |
5a16a542 KL |
1131 | advance Y by 1 |
1132 | next row | |
930d75f0 | 1133 | |
5a16a542 KL |
1134 | decrease Z by 1 |
1135 | next layer | |
1136 | ``` | |
1137 | ||
1138 | A terminal is free to optimize its rendering as it sees fit, so long | |
1139 | as the final screen output looks equivalent to the above method. | |
1140 | ||
1141 | ||
1142 | ||
930d75f0 KL |
1143 | Layers - Integration With Existing Sequences |
1144 | -------------------------------------------- | |
1145 | ||
1146 | Sequences that insert characters/lines, delete characters/lines, or | |
1147 | modify larger regions are changed to act upon multiple layers as | |
8af51dab KL |
1148 | defined below. By default, MSL (Manipulate Single Layer) is |
1149 | off/unset, and Z is 1, so if the application never changes MSL or Z | |
1150 | then these sequences will produce the same visible output as a | |
1151 | terminal without layer support. | |
930d75f0 KL |
1152 | |
1153 | A terminal is not required to support all of these sequences; however, | |
1154 | for those sequences it does support, if it supports the layers feature | |
1155 | then the sequences must behave as shown below: | |
1156 | ||
1157 | | Sequence | Command | Additional behavior | | |
1158 | |------------|-------------|------------------------------------------| | |
203cfc23 KL |
1159 | | BS (0x08) | Backspace | Only current layer affected if MSL=on | |
1160 | | DEL (0x7F) | Delete | Only current layer affected if MSL=on | | |
1161 | | IND (0x84) | Index | Only current layer affected if MSL=on | | |
1162 | | RI (0x8D | Reverse Index | Only current layer affected if MSL=on | | |
930d75f0 KL |
1163 | | ESC # 3 | DECDHL | Cells on all layers always affected | |
1164 | | ESC # 4 | DECDHL | Cells on all layers always affected | | |
1165 | | ESC # 5 | DECSWL | Cells on all layers always affected | | |
1166 | | ESC # 6 | DECDWL | Cells on all layers always affected | | |
203cfc23 KL |
1167 | | ESC # 8 | DECALN | All layers > 1 cleared; Z, MSL, TFT, TBT reset to default | |
1168 | | ESC 7 | DECSC | Also store Z, MSL, TFT, TBT | | |
1169 | | ESC 8 | DECRC | Also restore Z, MSL, TFT, TBT | | |
1170 | | ESC c | RIS | All layers > 1 cleared; Z, MSL, TFT, TBT reset to default | | |
1171 | | CSI @ | ICH | Only current layer affected if MSL=on | | |
1172 | | CSI J | ED | Only current layer affected if MSL=on | | |
1173 | | CSI K | EL | Only current layer affected if MSL=on | | |
1174 | | CSI ? K | DECSEL | Only current layer affected if MSL=on | | |
1175 | | CSI L | IL | Only current layer affected if MSL=on | | |
1176 | | CSI M | DL | Only current layer affected if MSL=on | | |
1177 | | CSI X | ECH | Only current layer affected if MSL=on | | |
1178 | | CSI M | DL | Only current layer affected if MSL=on | | |
1179 | | CSI P | DCH | Only current layer affected if MSL=on | | |
930d75f0 | 1180 | | CSI R | DECSTBM | Cells on all layers always affected | |
203cfc23 KL |
1181 | | CSI $ t | DECARA | Only current layer affected if MSL=on | |
1182 | | CSI $ v | DECCRA | Only current layer affected if MSL=on | | |
930d75f0 | 1183 | | CSI x | DECSACE | Cells on all layers always affected | |
203cfc23 KL |
1184 | | CSI $ x | DECFRA | Only current layer affected if MSL=on | |
1185 | | CSI $ z | DECERA | Only current layer affected if MSL=on | | |
930d75f0 | 1186 | |
40e3b486 KL |
1187 | (( TODO: add many more to the above table... )) |
1188 | ||
930d75f0 KL |
1189 | The VT52 sub-mode commands: |
1190 | ||
1191 | | Sequence | Command | Additional behavior | | |
1192 | |------------|-------------|------------------------------------------| | |
203cfc23 KL |
1193 | | ESC J | ED | Only current layer affected if MSL=on | |
1194 | | ESC K | EL | Only current layer affected if MSL=on | | |
930d75f0 KL |
1195 | |
1196 | ||
1197 | ||
1198 | Layers - Use With Multiplexers | |
1199 | ------------------------------ | |
1200 | ||
1201 | Layers are inteded to provide a means for multiplexers to pass on the | |
1202 | job of multimedia support to the "outer" or host terminal. The | |
1203 | proposed mechanics of that is outlined in the pseudo-code below: | |
5a16a542 | 1204 | |
930d75f0 KL |
1205 | ``` |
1206 | for each inner terminal in descending order from maxZ to minZ: | |
5a16a542 | 1207 | |
930d75f0 | 1208 | emit CUPZ(inner terminal Z, inner terminal Y, inner terminal X) |
5a16a542 | 1209 | |
930d75f0 | 1210 | draw inner terminal text with standard VT100/VT102/xterm sequences |
5a16a542 | 1211 | |
930d75f0 KL |
1212 | for each multimedia sequence emitted by the inner terminal: |
1213 | emit CUP(inner terminal Y, inner terminal X) | |
1214 | emit multimedia sequences to outer terminal | |
1215 | next multimedia sequence | |
5a16a542 | 1216 | |
930d75f0 KL |
1217 | decrease Z by 1 |
1218 | next inner terminal | |
1219 | ``` | |
5a16a542 | 1220 | |
930d75f0 KL |
1221 | The method above may not be effective for complex multi-terminal |
1222 | screen layouts, but is hoped to work well for many simple cases. | |
5a16a542 | 1223 | |
5a16a542 KL |
1224 | |
1225 | ||
930d75f0 KL |
1226 | Layers - Examples |
1227 | ----------------- | |
5a16a542 KL |
1228 | |
1229 | ||
5a16a542 KL |
1230 | |
1231 | ||
930d75f0 KL |
1232 | References |
1233 | ---------- | |
5a16a542 | 1234 | |
930d75f0 | 1235 | * xterm control sequences: |
5a16a542 KL |
1236 | |
1237 | ||
930d75f0 | 1238 | * ECMA-48: |