Preface

How To Use This Manual

Typewriter Font

Additional Software Required

APIs for Development: Xlib, Xt, and Motif

Xlib Video Extension

X Toolkit: Xt Video Widgets

Sample Applications and Source Code

Sample Applications: VideoTool, MovieTool, etc.

Source-Code Example Programs

Upgrade and Cross-Platform Differences

Differences between Framebuffer and Overlay Cards

Differences in Recent Releases

Adding Support for Zoom_2X

Using Shared Memory

Using Multiple Video Cards

Xlib Enhancements

Xt Enhancements

MovieTool Movie File Structure Enhancements

Porting Issues and Cross-Platform Development

Making Upgrades and Ports Easier: imake

Porting from Video Framebuffer to Overlay Cards

Porting from Video Overlay to Framebuffer Cards

Supporting Audio for Sun and HP

Shared Libraries

Porting between Specific Platforms

Sun-Framebuffer-Only APIs

Additional Software Required

X Toolkit Requirements

• All developers need Xlib: You must have a developer's release of Xlib for any development using the Video Development Environment. You will need Xlib with both libraries and include files.
• Xt video widget users need Xlib and Xt: If you are using the Xt video widgets, you must have developer's releases of both Xlib and X Toolkit (Xt). You will need Xt with both libraries and include files.
• Motif users need Xlib, Xt, and Motif: If you are using Motif, you must have developer's releases of Xlib, X Toolkit (Xt), and Motif. You will need OSF Motif version 1.2 or later.
• XView API users need Xlib and XView: (For Sun framebuffer cards only) If you are using the XView (Appendix E) APIs, you must have developer's releases of both Xlib and XView Toolkit. Please note the XView APIs do not support cross-platform development, and we do not plan to support them in future releases.
• X11R5 users need Xlib and Xt: (For Sun framebuffer cards, SunOS 4.1.x only) If you are developing on X11R5 (Appendix F), you must have developer's releases of both Xlib and X Toolkit (Xt).

In case you need to get Xlib and/or Xt, first try to get it from your system manufacturer. As an alternative, see the note below on how to get these toolkits. If you are developing on HP-UX 10.x, you will need Xlib/Xt from X11R6 (or X11R5). If you are developing on Solaris 2.5 or earlier, or HP-UX 9.x or earlier, you will need Xlib/Xt from X11R5. (When we release support for Solaris 2.6, that will support X11R6.)

• If you need to get a developer's release of Xlib or the X Toolkit (Xt), you can contact the Open Group at:

  Web: Use http://www.opengroup.org/tech/desktop/x/gettingx.htm to locate and connect to the source of your choice and then download the software. (Or look on the Parallax site, www.parallax.com/customer.)

  ftp: Use anonymous ftp to connect to ftp.x.org and download the software. (A list of mirror sites is listed on the Open Group web site, www.opengroup.org.)

ANSI C Compliant Compiler

• If you want to get an ANSI C compliant compiler from the Free Software Foundation/GNU, see GNU's web site for ftp mirror site information at www.gnu.org/order/ftp.html. One ftp mirror site is ftp.uu.net/archive/systems/gnu.

APIs for Development: Xlib, Xt, and Motif

Xlib Video Extension

The Parallax extension to Xlib allows full access to standard Parallax hardware video inputs/outputs and JPEG compression/decompression:

• Core Functions -- Manage video inputs and output. Control input settings (color, image size, etc.).
• JPEG Functions -- Manage the real-time JPEG-hardware compression/decompression of video, compressing video/graphics for disk storage or network transfers and decompressing these for display. Applications include movie capture and playback, and videoconferencing. (JPEG compression/decompression is only supported on XVideo with compression and PowerVideo.)
• Utility Functions -- Manage using multiple video cards in one system. Pause the application. Et al.

X Toolkit: Xt Video Widgets

Parallax Graphics has developed four Xt video widgets: VideoIn, VideoOut, JPEG, and TCWindow. Each widget has a full set of resources. These widgets exercise standard Parallax Graphics hardware features to provide video operations, such as:

• VideoIn Widget -- Manages video inputs, displaying video in an X window from a source connected to a Parallax video input port. Controls input settings (color, image size, etc.).
• VideoOut Widget -- Manages video output, exporting a bounded X display region of the screen as video to a device (such as a VCR) connected to the Parallax video output port. (Video output is only supported by XVideo cards.)
• JPEG Widget -- Manages the real-time JPEG-hardware compression/decompression of video, compressing video/graphics for disk storage or network transfers and decompressing these for display. Applications include movie capture and playback, and videoconferencing. (JPEG compression/decompression is only supported on XVideo with compression and PowerVideo.)
• TCWindow Widget -- Creates TrueColor windows.

Motif Convenience Functions

Parallax Graphics has developed two convenience functions to create cascading menus for selection of video input and video output: PlxCreateInputCascade() and PlxCreateOutputCascade(). Other supporting functions are also provided.

See Chapter 4 for full details.

Sample Applications and Source Code

Sample Applications: VideoTool, MovieTool, etc.

These applications are fully tested and released.

The source files for the application programs are located in $PARALLAX_HOME/src in the following sub-directories:

• These motif applications are not available for SunOS 4.1.x/X11R5.

Source-Code Example Programs

These sample programs are not as carefully built as the application programs, but do provide useful brief examples of how to use the Parallax development tools and hardware capabilities.

To use the sample programs:

• Look for a program that has some functionality you want to use (see summaries below). See how the program works (Multimedia User's Guide, Appendix C) and try out the program.
• Look at the source code for the sample program.

  The source files for sample programs are located in $PARALLAX_HOME/src in the following sub-directories:

Upgrade and Cross-Platform Differences

Recent releases of Parallax Graphics software include new capabilities (imake support, multi-card support), new development gadgets (more Xlib functions, Xt video resources, and sample programs), and other changes. This section discusses the differences and how to use some of the features. Here is a summary of differences between framebuffer and overlay and between different software releases:

• * Sun overlay and HP version 9.3 or later have only a sample program, xvshow, with hints on how to do graphics over video.

Differences between Framebuffer and Overlay Cards

Overlay Cards Have:

• Zoom: hardware enabled pixel replication of video (live, decompressed, or still images)
• The ability to send video output from a video window hidden behind non-overlay windows.

Framebuffer Cards Have:

• Real-Time MovieTool, its API, and its sample programs (rtvPlay, rtvRecord) on Solaris 2.5 only
• Special Effects
  • software switching between color and black and white video
  • software switching between normal and inverse video
  • software switching between genlock and freerun video
• Graphics over video sample programs (govLogo, govLogo2, govTest)
• Xt Enhancements: CustomDrawMethod - This is available for overlay cards, but since it uses graphics over video, we do not recommend using it for our overlay cards.

Differences in Recent Releases

• Shared memory for HP (already on Sun)
• Several video cards in one HP system
• imake support for Sun and HP
• Shared libraries for Sun and HP
• Multiple video operations in a window for HP overlay (already on Sun Framebuffer)
• More Xlib functions, Xt video resources, and sample programs on Sun and HP,
• MovieTool file format enhancements: portable audio, compressed audio, and variable Q factor,

XView Objects and MovieTool API: Phasing Out

• XView Video Objects (VIDEO_IN and VIDEO_OUT)
• XView clients and libraries
• XView-based MovieTool JPEG Movie API
  • The MovieTool file format is still fully supported and will continue to be supported. Just the XView API for MovieTool will be phased out.
• XView-based device control API and RS-232 support

Adding Support for Zoom_2X

Using the Zoom_2X capability requires some changes to how you use some of the Xt Widgets' X resources and Xlib extension functions. The differences are described in each resource and function description:

Zoom related functions:

XPlxGetZoomFactor()

XPlxSetZoomFactor()

Zoom and video input:

XPlxQueryVideo()

VideoIn widget: XtNdisplayRect

VideoIn widget: XtNsourceRect

Zoom and video output:

XPlxVideoScaleOutput()

VideoOut widget: XtNdestRect

VideoOut widget: XtNsourceRect

Zoom and JPEG compression/decompression:

XPlxGetCImage()

XPlxPutCImage()

XPlxShmGetCImage()

XPlxShmPutCImage()

JPEG widget: XtNcompressDestRect

JPEG widget: XtNcompressSourceRect

JPEG widget: XtNdisplayDestRect

JPEG widget: XtNdisplaySourceRect

Zoom and uncompressed images:

XPlxGetImage()

XPlxPutImage()

Using Shared Memory

• Sun video framebuffer cards--XVideo, PowerVideo, and MultiVideo
• Sun video overlay cards--XVideo Xtra
• HP video overlay cards with version 9.3 or higher--XVideo700, PowerVideo700, and MultiVideo700

Shared Memory for Compression and Decompression

Parallax supports shared memory only for compressed video and images. For uncompressed shared memory operations, programmers must continue to use the existing XShm...() MIT-SHM shared memory extension routines.

The new Parallax shared memory routines follow the MIT-SHM calling conventions and semantics. If existing code calls MIT-SHM routines for JPEG shared memory operations, the code should now call the Parallax versions of those routines. The following is a list of all affected functions:

MIT-SHM	Parallax Graphics
XShmDetach	XPlxShmDetach
XShmAttach	XPlxShmAttach
XShmGetEventBase	XPlxShmGetEventBase
XShmQueryVersion	XPlxShmQueryVersion

Xlib Extension and Shared Memory

• XPlxShmCreateCImage()
• XPlxShmGetCImage()
• XPlxShmPutCImage()

Xt Video Widgets and Shared Memory

Using Shared Memory on HP

This garbage data does not cause a problem unless you use software decoding. If you display JPEG and JFIF compressed video and images only with Parallax hardware decompression, you can ignore this garbage data. If you ever do software decoding such as displaying a still JFIF with third party applications that use software decompression, or converting JPEG movies to other digital movie formats, then you must remove the garbage data.

The four bytes of garbage data would be located in the digital movie or image file right after LOAD_JPEG. In Parallax software version 9.3 or higher for HP, MovieTool and VideoTool correctly remove this garbage data (scomp does not), so you can look at VideoTool and MovieTool as examples.

If you are using Parallax's Xlib extension:

You should write your application to remove the garbage data for JFIF and JPEG still images and for JPEG movie files. After using XPlxShmGetCImage(), you must add some code or example below) to remove the garbage data before writing a still image or movie file.

If you are using Parallax's Xt widget:

The Xt widget will automatically remove the garbage data before saving a JFIF still image.

You must write your application to remove the garbage data for JPEG still images and JPEG movie files. After using XtNGetNewImage(), you must add some code or see the example below) to remove the garbage data before writing a JPEG still image or movie file.

Here's an example showing how to remove the garbage data:

For more information, see (Xlib), (Xt), and Appendix C.

Using Multiple Video Cards

You can look at the applications VideoTool and MovieTool in HP version 9.3 or higher to see how to display and manipulate video through multiple video cards. Note that VideoTool and MovieTool software can select video from up to three video cards (up to six inputs if the cards are XVideo700's), BUT the VideoTool software is limited to displaying only two live video inputs at once, and MovieTool can display only one. For displaying more inputs at once, the work-around is to open multiple VideoTool sessions--displaying each video input in only one window to avoid conflicts. VideoTool and MovieTool can also select output to any XVideo card in the workstation, but can send only one output at a time. And keep in mind that workstations generally support one audio input channel and one audio output channel at a time.

• Other platforms/cross-platform: If you build your application on version 9.3 or higher for HP using this multi-card capability, you can recompile your application and use it on Sun for single card systems. You can't use the multi-card capability on Sun.
• One card per window: A window can only be attached to one video card at a time. For instance, you can't set up video input from one card and video output to another card at the same time from the same window.

XPlxVideoInputSelect()

16 bits of the channel parameter. For example, to display video from the "Video
In Composite 2" wire on the second Parallax Graphics video card, use a command similar to this:

XPlxVideoOutputSelect()

• In the overlay scenario, you cannot output regions that are not being displayed with the card that is doing the output. That is, you cannot use video card 0 to output information being displayed on video card 1.

Xlib Enhancements

Xt Enhancements

TCWindow

The new widget and resources are described in Chapter 3.

Motif Enhancements

MovieTool Movie File Structure Enhancements

The current version of MovieTool (Sun Solaris 2.5 and HP version 9.3 and 10.10) captures movies in JPEG_VERSION_2 and then converts them to JPEG_VERSION_3 if you use MovieTool to trim frames off the beginning or end of the movie. MovieTool does not implement JPEG_VERSION_4, so there is no example code for the JPEG_ VERSION_4 enhancements.

MovieTool uses the lowest common version of the movie file structure to ensure cross-platform compatibility with older versions of Parallax MovieTool and videotool, and for playback by Sun XView. XView plays back JPEG_VERSION_2 movie files.

The changes to the MovieTool movie file structure are indicated by the updated "movie.h" file with new operands, and by new attributes in the movie file header. Most of these changes (JPEG_VERSION_4) are not implemented in MovieTool at this point, so you will have to work without example code. The new capabilities include:

• Trimming frames off the beginning and end of a movie. The "trimmed" frames are not played back, but remain in the movie file to eliminate the need to re-record the movie. See attributes "firstValidFrame" and "lastValidFrame" (Appendix C) and example code in MovieTool.
• Changing the baseline audio volume control in a movie file. See attribute "volbase" (Appendix C).
• Changing the Q factor within a movie file (changing one time, several times, or every frame). Varying the Q factor can be useful for some methods of synchronization and bandwidth-management. See operand "NEW_QFACTOR" (Appendix C).
• Storing application-specific data. See operands "APP_DATA_0" to "APP_DATA_3" (Appendix C).
• Using compressed audio data. See operands "LOAD_COMPRESSED_AUDIO_0" to "LOAD_COMPRESSED_AUDIO_3" (Appendix C).

Porting Issues and Cross-Platform Development

Making Upgrades and Ports Easier: imake

Not all platforms provide imake. If you do not have imake on your system, you can get the source and build it yourself. The source is available from ftp.x.org/pub/R5.

Parallax Graphics now ships an imake template file that makes writing an Imakefile for video applications easy. The template file is in the $PARALLAX_HOME/include directory, named "Parallax.tmpl". This template file should be copied into the config directory (usually in /usr/X11/lib/config).

The easiest way to write an Imakefile is to copy one from our source tree and modify it for your needs. Be sure to copy an Imakefile from an appropriate part of the tree. For example, if you are writing an Xlib application, start with $PARALLAX_HOME/src/xlib/example/Imakefile. If you are writing a motif application, start with $PARALLAX_HOME/src/motif/VideoTool/Imakefile.

For more information about setting up Imakefile support for specific Xt widgets, see Chapter 3 in this manual.

• For more information about imake, you can refer to a general manual on imake, such as O'Reilly & Associates' Software Portability Using imake.

Porting from Video Framebuffer to Overlay Cards

Multiple Video Operations in a Video Window: xv Ports

On Sun overlay version 1.0 and HP version 9.2 overlay, you can get video in and video out to appear as if they are in the same window by using the root window for the output operation.

The reason for this problem is related to xv ports. xv ports are allocated for each window doing video operations. It is possible to get the same xv port attached to more than one window. This can cause erratic operation if an xv port is shared between live video and a decompression operation.

To avoid this problem, be sure to set up your live video first. In applications that display live video and perform compression and decompression, you should perform a XPlxVideoInputSelect() before any calls to XPlxPutTable().

• Sun overlay version 1.1 and HP version 9.3 and higher: This problem was corrected in Sun overlay version 1.1 and HP version 9.3 or higher so that more than one video operation can be run from one window/xv port at a time. For example, you can display live video in a window and simultaneously send video output from the same window.

Graphics Over Video

It is possible to do graphics over video on video overlay cards. Unfortunately, we do not currently have an API that handles graphics over video on video overlay cards to speed up your work. If you want to use graphics over video for video overlay cards and you are using HP version 9.3 or higher or XVideo Xtra, look at the xvshow.c file as an example.

Adding Support for Zoom

Overlay cards offer hardware-enabled zoom. See here to find out where you need to set up zoom.

Porting from Video Overlay to Framebuffer Cards

Software for Video Overlay Cards Runs on Framebuffers

Supporting Audio for Sun and HP

Shared Libraries

Solaris 2.4 or earlier (framebuffer), HP version 9.2: These releases did not contain shared libraries. If you are writing an application that you intend customers using these releases to use, you must link statically.

Porting between Specific Platforms

Sun Overlay to/from HP Version 9.3 or Higher

• Support for multiple video cards is available on HP only; see here
• Multiple video operations in one window work on Sun overlay (Xtra) version 1.1 and HP version 9.3 or higher but not on Sun overlay version 1.0 or HP version 9.2; see here
• Audio support (Sun versus HP, always an issue in cross-platform porting); see here
• Software that's available in our HP release but not in our Sun release: MPEG conversion, H.261 conversion. Also the ImageMagick convert and display utilities are included in the HP and Sun overlay releases but not Sun framebuffer. ImageMagick is available free however (contact tsupport@parallax.com).

HP Version 9.2 to/from HP Version 9.3 or Higher

• Support for multiple video cards is available on HP only, and fully implemented (supporting compression, decompression, and video output) only on HP version 9.3 or higher; see here
• Multiple video operations in one window work on HP version 9.3 or higher but not on HP version 9.2; see here
• imake is fully supported on HP version 9.3 or higher (imake is just partially supported on HP version 9.2)
• Shared memory is available on HP version 9.3 or higher; see here
• Shared libraries are available on HP version 9.3 or higher; see here
• New Xlib functions, Xt resources, and Motif functions are available with HP version 9.3 or higher, see here.
• A graphics over video example program (xvshow) is available on HP version 9.3 or higher
• Uncompressed images (see notes below)

Uncompressed Images

In our HP 9.2 release, we overloaded the XGetImage() and XPutImage() functions to display an image into Parallax video memory if certain conditions were met. These conditions are as follows:

• An XvPort must be attached to the window before the Put/Get function is called. This is generally accomplished by temporarily starting video in the window, and then turning the video back off.
• For the XPutImage() call to succeed, the XImage must have a 32-bit depth. The lower 24 bits represent the color, and the upper 8 bits should be zero.
• The depth of the window must be the default depth. That is, you cannot create a 24-bit TrueColor on an 8-bit root window, and get/put XImages into Parallax video memory.

Shared Memory

Our 9.2 release did not support shared memory. If you are writing an application that uses shared memory and you will be supporting customers having our 9.2 and 9.3 release, you should seriously consider providing non-shared memory support. scomp is a good example of how to detect if shared memory is available. If your application must use shared memory, all of your customers will need our 9.3 release.

Shared Libraries

Our 9.2 release did not contain shared libraries. If you are writing an application that you intend customers using our 9.2 release to use, you must link statically.

Multi-board Support

Our 9.2 release had limited multi-board support. Applications could get live video to work on either of 2 Parallax video boards installed in the system.

Our 9.3 release has enhanced multi-board support. We now support up to 3 Parallax video boards installed in the same system. One instance of an application can now use all functions of all installed video boards.

If your application requires multiple Parallax video boards and some of your customers are using 9.2, they must upgrade to our 9.3 release.

Multiple Video Operations in the Same Window

Our 9.2 release did not allow more than one video operation to occur at the same time in one window. For example, a 9.2 application cannot create a window, display live video in it, and then output the live video from the same window. To achieve this particular effect, the application would have to output from the root window the region bounded by the video window.

Other examples of multiple video operations in a window include:

• live video and JPEG decompression
• output and JPEG decompression

Our 9.3 release fixes this problem. You can now do as many different operations in one window as you like. Here are the conditions:

• You must use a different gc for each separate operation. For example, if you are displaying live video and sending the video contents to video output, you must have a gc to handle live video, and a separate one for output.
• You are limited by the video hardware capabilities. For example, with our XVideo-700 board you cannot display live video, output video, and decompress at the same time. The decompression will cause either the live video or output to freeze momentarily.

Sun-Framebuffer-Only APIs

Real-Time MovieTool for Solaris 2.5

If you are using Solaris 2.5 and Parallax video framebuffer cards with compression, and if you want to make your application capture and play back frame-accurate movies to/from a local disk in real-time at near-broadcast quality (full motion, full size, 24-bit TrueColor), you should try the Real-Time MovieTool (RTMovieTool) API.

The RTMovieTool API is currently available only on the Sun platform, for Solaris 2.5 and video framebuffer card users (not XVideo Xtra). RTMovieTool requires hardware compression (XVideo-SVC-VIO or PowerVideo) and a SPARCstation 5, 10, or 20 with at least 32 Megabytes of ram.

To see RTMovieTool at work, read your Multimedia User's Guide's Chapter 5 and test drive RTMovieTool.

To bring these capabilities into your own application, read Appendix D: Real-Time MovieTool JPEG Movie API and File Structure (Sun Solaris 2.5 Only) in this manual. Appendix D includes code examples and a documented program, troubleshooting, a complete reference section, a description of the RTMovieTool file format, and sample programs to extract information from RTMovieTool movie files.

• Regular MovieTool may capture almost every frame at full size and default Q factor if your machine is fast enough (for instance XVideo Xtra on an UltraSPARC captures 28-30 frames/second), BUT some frames in the movie may be torn. Regular MovieTool can't guarantee capture of every frame (and every interlaced field) like RTMovieTool does.

XView Toolkit: XView Video Objects and MovieTool API

Developers using the XView Toolkit from Sun can use two XView video objects (with a full set of object attributes): VIDEO_IN and VIDEO_OUT. See Appendix E for details.

If you are using XView, you can also use the MovieTool JPEG Movie API for creating movie-file authoring applications. The API provides a library of functions for recording JPEG-compressed movies in the MovieTool movie-file format and for playing these movies back in near real-time with hardware compression and decompression. See Appendix E for details.