von W. Connolly, J. Gait, J. R. West - Sony Pictures High Definition Center (aus 1993 !!)

There has been a dramatically increased demand for the application of electronically generated high resolution digital imaging techniques to motion picture and television programs. A variety of newly established digital image production and post-production centers have taken their place next to the more traditional film* optical houses. An extraordinarily wide variety of techniques are being developed to supplement and, in some cases, to supplant optical techniques in film production.

The Sony Pictures High Definition Center, which originally opened its doors as Sony High Definition Facilities in October 1991, is currently applying High Definition digital imaging processes to feature film production and post-production. The Center has chosen a particular direction to take, one which exploits the interaction between real-time digital image processing and creative talent. Directors, cinematographers and special effects supervisors have created complex, multi-layered visual effects at the Center's image compositing room, and have viewed the results immediately in real-time. Final decisions can be made after viewing the scenes in a newly completed screening room which accommodates electronic as well as film projection using a novel design. The results of this unique interactive process have been incorporated into three recent feature film releases: Disney Studios' "Honey I Blew Up the Kid," Warner Brothers' "The Power of One" and MCA/Universal's "Trespass."

The Center uses the SMPTE 240M/260M Production Standard which is based on 1,125 vertical lines and a 60 Hertz picture repetition rate. SMPTE 260M specifies the digital representation of the analog 240M standard and requires an information transfer rate in excess of one Gigabyte per second. This is the highest bit rate which can be recorded and displayed in real-time with current equipment implementation. In computer terminology, the system handles IK by 2K resolution in real-time. The aspect ratio of the scanned area is 16 units by 9 units, which is virtually identical to the aspect ratio of the vast majority of Hollywood feature film releases. This standard also provides for color resolution which is ten times current television systems and the ability to record and display luminance detail which is five times current television systems. Compact Disc quality sound accompanies the picture information.

Today, materials to be used in the creation of a feature film come from a variety of sources. In addition to those materials which are photographed in various film formats, many are generated on a computer platform; and some are photographic sequences which require image manipulation. To accomplish the image processing necessary to merge these elements, all are brought to a common form known as an "electronic intermediate." There are a variety of digital electronic intermediate file formats. Most take the form of a digital representation of the red, green, and blue records of each frame. SMPTE 260M uses a variation which consists of a digital record of the luminance and color difference signals for each frame. Once in the digital domain, the processes applied to the image, such as multiple replications, take place without the degradation associated with similar analog signal processes. Thus, complex layering processes which can have ten or more elements can be accomplished without the resolution losses associated with analog film opticals.

Film elements are converted to electronic intermediate form by means of a scanner. At the High Definition Center at Sony Pictures a unique approach has been taken. A single Hyper-HAD™ CCD containing an area array of 2.2 million pixels is used to scan the Red, Blue and Green contents of each film frame in turn. The optical system includes a Xenon light source, sequential color dichroic filters, a mirror box integrator and a 95mm Nikkor printing lens. The CCD camera has provision for multiple axis adjustments including rotation. Pin-registered transfer takes place at two frames per second. VistaVision, 35mm, and 65mm formats can be accommodated; provision for 16mm film format is being added.

Two modes of operation have been incorporated into the scanner design. In the "Telecine Mode," the system is operated like a conventional telecine. Under the client's supervision, RGB gains, black level controls and gamma corrections are adjusted to create a pleasing video image which may be quite different from the film print image that would result from printing from the original negative. In the second or "Intercut Mode," all efforts are applied to generating an image which is capable of being seamlessly intercut with a film camera original image, one that has not been through an electronic processing stage. In this mode of operation, gamma and colorimetry settings are chosen from a library which allows the High Definition image to be matched to a film strip projected image. This process is almost identical to a motion picture laboratory's use of an electronic film analyzer, and results in an image which when taken back to film through the Electron Beam Recorder (to be described later), closely matches the dynamic range and colorimetry of the original film clip.

As mentioned earlier, computers are increasingly becoming a source for images to be included into feature films. The Center uses a Symbolics XL-1200 platform for generating computer graphics and computer animation sequences. The workstation operates in conjunction with an HDDF-500 framestore which holds 12.8 frames of digital High Definition images in RAM storage. This allows the computer to access more than four seconds of information randomly. For (non-real-time) retouching of images and for frame-by-frame Rotoscoping, this allows the retouch artist to access frames one by one and replace the frame with absolutely no image degradation. As is often the need in animation work, the four-second sequences can be cycled in real-time to judge the fluidity of motion.

SCSI interfaces have been written for a wide variety of computers for the HDDF-500 digital framestore. By working with computer vendors to complete the interfaces, IBM and Apple personal computers, Silicon Graphics and Sun workstations and Symbolics Graphics systems can be accommodated.

Computer images delivered to the Center in Exabyte™ format tape cartridges can be converted to SMPTE 260M by a highly automated process which was developed internally. This technique can be used to view computer animation work-in-progress using HDVS projection systems without committing to the (far more expensive) conversion to 35mm film for screening. Of course, this technique can also be used to integrate computer animation sequences into material destined later for conversion to 35mm film by means of the Electron Beam Recorder to be described below.

This process of exchange of images among computers, SMPTE 260M and film is reversible. The process has been put to practical use in preparation of theme park rides which had their origin in feature films and were transferred from the original film to SMPTE 260M, to computer platform for animation processing and back to High Definition laser disk for electronic projection at the theme park.

Once the electronic intermediate is created from any of the various sources ( film, computer image, or live camera), the sequences are then stored on video tape recorders for real-time playback. The Center is equipped with HDD-1000 VTRs which operate at an astonishing information transfer rate of 1.2 Gigabits per second (to meet the requirements of SMPTE 260M). The information stored on these recorders can be viewed in real-time, edited into new sequences, be A-B rolled for compositing or undergo other common image manipulation, such as digital video wipes, fades or digital video effects and rerecorded. Eight channels of compact disc quality digital sound, a monitor channel and time code are also recorded on the one-inch tape.

The Digital Multiple Effects unit at the Center is a prototype, not yet committed to product form, and incorporates most of the common features such as sizing of images, repositioning, transportation, rotation, et al. The unit is operated under computer control with sequence of operations repeatable from memory, and fully meets the SMPTE 260M specification.

An HD Ultimatte 6™ is used for the all-important process of image compositing. This device, operating at a bandwidth consistent with the electronic intermediate elements created from the processes above, makes possible the combining of multiple elements (or layers) of foreground and background elements into a composite image with subtlety and completely without intrusive matte lines. In addition to clean mattes, shadows from foreground images can be made to fall upon background plates --whether the plates were created from stock footage, recorded for the purpose, or were created from matte paintings - an effect not possible with film optical processes.

Also, since the images to be processed are in digital electronic intermediate form - many derived from multiple generations of image processing - image degradation is minimal. Far less degradation from multiple layering of images takes place than is common in correlative film optical processing. This is particularly true when eight or more layered elements is involved, as is frequently the case in the more complex effects of modem films.

Once the sequence of electronic intermediate images is combined under the creative control of the director, cinematographer or special effects supervisor, the finished product can be converted to 35mm film by means of an Electron Beam Recorder. This method of transfer to film was chosen over the more common cathode ray tube or laser beam methods to preserve the resolution of the input image. The EBR does not have a gaussian spot exposure profile when exposing film in a vacuum chamber; therefore, film scans at 2000 picture element resolution compare favorably with scans of twice that resolution from CRT or laser film scanners.

An HDC-1000 VTR recording of the final cut is played to the EBR at one frame per second to create an RGB black-and-white fine grain film image of each frame sequentially. This transfer rate is orders of magnitude faster than some CRT scanners currently in operation. This RGB film record is similar in function to the film YCM separation negatives which are currently the means of archival storage of color feature films. An added advantage for the EBR product is that the frames are recorded sequentially one one film reel for the separation records rather than on separate reels, avoiding the common problem of differential shrinkage of the color separation frames during storage.

From the RGB EBR black-and-white master, a step printer is used to create a color negative, an interpositive or, in some processes, envisioned directly to a printing negative from which release prints can be struck directly. To date, both negatives and interpositives have been created at various contrasts to match adjacent sequences which came directly from film cameras.

Feature film directors, cinematographers, special effects supervisors and editors are accustomed to screening the results of their work on a large-screen presentation. A screening room was constructed at the Center which permits viewing of High Definition electronic images and 35mm film projected images in very high quality, at matched brightness levels and with a cinema-like sound system. Considerable architectural, acoustical and optical design effort has resulted in a highly satisfactory result.

SMPTE film projection standards for screening rooms was the basic design criteria. To match those standards, a screen curved in both horizontal and vertical directions was chosen to provide a brightness uniformity not possible with a flat screen. The screen material also gives a gain sufficient to accurately match the electronic projection system to the open-gate standard for projected film.

A six-channel sound system was installed (now being expanded to eight channels) to match current cinema practice. Since the screen is not perforated as is usually the case for film projection facilities, the speaker system was designed to match the left-center-right cinema sound system with an acoustic image which matches that of speakers which would ordinarily be placed behind a perforated screen.

The optical design of the two projectors is such that the image can be vertically split between the electronic projection and film projection. This is particularly useful to the cinematographer who may wish to view images before and after the electronic intermediate was processed through the Electronic Beam Recorder to produce film.

The story line of this motion picture centers around a child whose scientist-father caused him to grow progressively over time from two feet in height to 120 feet in height. Disney Studios submitted the film elements for each of the many effects sequences to a variety of film optical and electronic digital image processing effects companies. Sony Pictures High Definition Center completed more than twenty effects sequences, eleven of which were in the final release print.

In the final stages of the story line, the child walks down the main thoroughfare of Downtown Las Vegas, Nevada, towering over most of the buildings. In some effects sequences, computer-generated effects had to be added to the film foreground and background elements to create electronic explosions and sparks caused by the child knocking over lighted neon signs and knocking over telephone booths with full-sized automobiles which he handled as though they were toys. Considerable digital image processing using all the signal processing facilities at the disposal of the Center were necessary to make the effects seem realistic.

This feature film is a drama and not, as in the sense of "Honey, I Blew Up the Kid," an effects film. The story line is about Apartheid in South Africa and was filmed almost entirely on location.

In the opening sequence, the screen pans to a map of South Africa under a prologue created in the computer, setting the time and place, rolling vertically. The camera then dissolves through to a scene in the African Veldt with a series of long dissolves depicting the birth of a child in an African Veldt farmhouse. Using film optical techniques, it was extremely difficult to get the relative densities of the very long dissolves exactly right. By using electronic techniques, the director and cinematographer could create the dissolves and view the results in real-time. The interaction of the creative team with the electronic equipment permitted a precision of image density that would have required many repetitions of A-B-C roll printings and waiting for film development of each before the timing was satisfactory.

Two scenes, one interior and one on a veranda, were shot against the hot African sun. In the interior shot, a midwife hands a newborn child to its mother. The first half of the shot, against an open window, is too dark. However, as the child is handed to the mother against another window, the lighting is perfect. Through electronic means, we were able to raise the foreground image light values against the bright window and use a 150-frame dissolve of that portion of the shot back into itself to cause a seamless transition as the mother takes the child.

Another scene contains an elephant charging a young boy, photographed with the elephant traveling first foreground to background and with a camera reversal with the elephant approaching the child. When these two takes were cut together, the elephant seemed to be moving too fast going away from the camera and too slowly approaching the camera. Electronic digital processing was used to both slow down and speed up the elephant charge so that the movement looked smoother through the camera cut.

In a third scene, a battle has taken place at night in an African village. As is frequently the case, the photography ran over schedule and the sun began to rise. When the scenes were intercut, the brightness of the skies did not match. The director and cinematographer gave instructions to the digital effects team members which allowed image processing to match the night sky throughout this complex and pivotal scene.

Finally, after audience preview reaction, it was decided to modify the closing scene of the movie. Since the opening date had already been fixed, there was insufficient time to return to location to shoot a new ending. Instead an ending was shot with the two principal actors walking over a hill in California. This scene was composited with a computer-generated rising sun matched accurately to the 600mm film lens used for the foreground element. It was necessary to create shimmering of the edges of the sun and cloud changes since this closing shot was held over the epilogue and closing credits for a duration of some seventy seconds of screen time.

The above scenes, among many others which underwent digital electronic processing, were excellent examples of post-production digital processing for other than "effects" scenes.

Next on our agenda is the use of the electronic intermediate process for the post-production of all types of feature films. Major films will continue to be shot on film for the foreseeable future; however, by transferring the camera original negative to High Definition, the entire post-production process can be done electronically, as is the case for major television drama which is originated on 35mm film. Large-screen projection of dailies and edited sequences with the quality of film print projection is now made possible electronically by High Definition projection systems. The camera original negatives are stored in the vault until the electronic post-production process is completed. The High Definition dailies viewing copies are down-converted to NTSC for off-line editing using one of the many highly-refined systems commonly in use. Once a final edit decision list is completed, it is used to automatically conform the High Definition "dailies" copies into an edit master. The edit master can be used for preview audience viewing in one or more versions if so desired; and once finally approved, can then be converted to release prints via the EBR process. Alternatively, the High Definition edit master can be used to create a negative cutting list for the camera original negatives.

This process must be tested against current work practice, speed of completion and cost to prove its viability. All indications are, at the outset, that considerable gains can be made.

New systems are under development to explore further applications of digital image processing for archival restoration and preservation using massively parallel processing computer platforms which have been developed by Sony. Such applications as scratch removal, grain reduction and image enhancement are now under investigation.

The use of digital image processing has proven to be effective in the production of several major motion pictures including: Disney Studios' "Honey I Blew Up the Kid," Warner Brothers' "The Power Of One" and MCA/Universal's "Trespass" at Sony Pictures High Definition Center, located in Culver City, California, USA. Final release footage will be shown during the presentation of this paper, with a tutorial description of how the effects were accomplished.

SMPTE 260M has been established as a viable electronic intermediate in special effects applications and post-production image processing for feature films, and by means of conversion to 35mm film through the Electronic Beam Recorder as an effective means of creating negatives for feature film release.

New applications for SMPTE 260M as an electronic intermediate are being developed for the post-production of feature films. Further applications are under study.