Prototype breadboard for digitsing faces

This object represents the development of video games through the work of Ralph Baer, referred to as the “father of video games”, who also pioneered the Brown Box and Magnavox Odyssey. Institutions such as the Smithsonian Museum hold similar objects to highlight the impact and significance of Ralph Baer on the development of home video games. - Arieh Offman, Curator - Video Games, ACMI Renewal

**

Updated information from Mark Baer and Ralph Baer’s engineer/collaborator Bob Pelowitz sent to Megan Taylor and Arieh Offman 7/02/2020:
Here’s an update and clarification regarding the purpose of the breadboard sent to the ACMI from the Ralph H. Baer Trust.

Initially, our (Bob Pelovitz and I) considered opinion was that the breadboard appeared to be associated with a project for digitizing faces and placing them into video games. As ubiquitous as that process is today, it was unknown back when Ralph first considered the concept. (I can send a copy of that patent by separate email. I can also provide further information concerning that patent should you desire it).

After further consideration, mostly from Bob - who worked with Ralph for decades on many of his projects - along with a look at some of Ralph’s notes and recorded recollections, we have concluded that your breadboard was a significant part of a project know as the “Four-Corners” (4-C) program. Bob actually started working with Ralph about a year before the program began, so he witnessed it directly.

The basic idea of 4-C was the creation of a branching interactive video entertainment system that allowed for different applications, most notably testing/quizzing and similar activities. Below the break, I have included Ralph’s own words. Please note that this is forwarded to you for museum purposes only, courtesy of the RHB Trust all rights reserved, unless otherwise agreed to in writing. That said, I’d be excited to see you feature this project/concept and I (and no doubt Bob, copied on this email) would be happy to continue this conversation.

As you will see, this was quite a forward looking novel concept and process in its time. I hope you find this information helpful and look forward to hearing from you on all matters. Let me know if you have any questions.

• Mark

• The Four-Corners (4-C) / TIM program

(This was) my latest attempt to create a practical interactive video entertainment system with “branching” capability, using a linear pre-recorded motion-video scenario medium such as a VCR tape.

Here’s the problem I tried to solve in nutshell: In an interactive video presentation, say a quiz program similar to Jeopardy, there will be multiple instances where the narrative or quiz should stop momentarily so the viewer can be individually engaged by “answering questions” or otherwise interacting with the presentation. By way of another example, suppose the viewer is a student taking a course on basic chemistry via videotape. Having seen a particular laboratory experiment executed on screen, it makes sense to stop and quiz the student now, perhaps using multiple-choice questions. The student-viewer responds by pressing buttons on a hand control or a keyboard, or uses a light pen.

So far, so good. Now comes the hard part:

Depending upon the answers the student keyed in, the show must progress, or branch in accordance with the answer that was chosen. I decided to finesse this branching problem by dividing the video screen up into four quadrants; each of these could display totally different video presentations. All but one of them would be covered – hidden from view – by a “window shade.” When the program required branching to one of these three new paths in response to the students answers, the current window would close and another, appropriate one would open and play back pre-recorded visuals that met the current need of the student. In other words, my 4-C system allowed me to branch three different ways.

Fine: So now we are in effect watching four different TV screens - small ones, to be sure - with only one of them normally in sight. We’ve licked the problem of how to branch visually in a linear (videotape) medium – remember: there were no DVDs back then. That’s great but it gets harder right away when you realize that those four “picture windows” need associated sounds…four tracks of audio. Ordinary video tape players have only two (L & R stereo) sound tracks. Obviously that wouldn’t hack it. We needed four audio channels to go with our four video presentations.

That was the next hurdle I had to overcome.

I came up with the concept (and several techniques) for nesting those four audio tracks within the video picture. Four narrow tracks of audio data were placed in the video signal at the left edge of the picture (a technique very similar to how audio tracks are placed on movie cinema films). These tracks would be hidden by the plastic frame around the CRT of a typical TV receiver or monitor.

TECHIE ALERT: Yes, for those of you who’ve gotta know: They were amplitude-modulated pulses, about a microsecond wide, placed right behind color-burst. The image nearby shows these tracks (white vertical bands) clearly, however they were placed within a part of the signal called “over-scan” that was hidden from view, in the final product.

That seemed like a straightforward solution to the problem of where to put an audio track for each of the four video windows. Unfortunately, it didn’t end there; there was another major technical hurdle to be surmounted. There is a gap between successive video frames (screen pictures) on videotape, unlike in movie film. That gap clobbered my nested audio data by producing a nasty rasping sound that I had to get rid of somehow. I came up with a neat system for overcoming those gaps and their resulting “pops.” It basically involved speeding up the sound recordings on each video frame to keep them away from the gap. During playback, these recorded sound signals were then “stretched” in time, played back slower, to make them meet seamlessly. Presto: No gap, no popping noises…problem solved!

A December 1984 video recording I made in the lab, having accomplished the nesting and extraction of four audio tracks, is shown below:

Several months went into that development job. Eventually I was able to record four-track audio data on my videotapes alongside the 4-C pictorials and then play it all back almost completely noise-free.

“We” turned out to be me for the next year – at least as far as building hardware was concerned. (My partner’s) part of the job was to produce a short videotape that would do a good job of demonstrating the 4-C method of branching in audio and video. In short order, he sent me a two-minute videotape he and his daughter had put together. It showed a blue hand puppet arguing with an off-stage female voice and soliciting the viewer’s opinion to exercise the “branching” modes of the system. (He) put on a thick southern accent for his performance. I still have the tape around somewhere.

• Side track – crowbar modulation

(I) demonstrated to him still another interactive video scheme. This one allowed me to overlay black-and-white graphics or alpha-numeric characters on any ongoing video motion scene. A good present day example would be the captions that modern TV sets let you view at the bottom of the screen. While that is a technical lead-pipe cinch today, it wasn’t then. TV sets had no video input jacks, so the only way you got to see something on the screen was to put an appropriate signal into the TV set’s antenna terminals (remember them?)…and that was that!

I came up with a scheme, which I called crowbar modulation; that method solved the problem with some very simple hardware.

TECHIE ALERT:It took advantage of the principle that in a U.S. analog TV system, a low level of received television signals at the antenna terminals produced a white image. This convenient fact led to the idea of crowbar’ing, i.e.: “choking”whatever TV channel signal you were tuned to in just the right way to put white letters or graphics anywhere on the screen, overlaid on the ongoing TV program.

My records show that I shipped that unit out on August 14 of 1984 (.)

• 4-C becomes an official development program

I started to design and build a new 4-C system in early August of 1984; my partner in this matter decided that this project was important enough to warrant a parallel effort in California. He contracted a third party, independent designer , to come on board. He was supposed to design and develop a digital encoder and decoder for 4-C; a device to place data on the videotape for use with interactive program segments. At the same time, negotiations with Texas Instruments began but was successful in landing a development contract in September. TI made an option payment of $250,000 to Western Technologies. We were off and running!

By December I was ready to produce the first completely functional 4-C videotape in my lab. My partner came to my place in the middle of January to look at progress. It all worked well. As usual, there were a variety of minor bugs. We decided on what I still absolutely had to do before shipping the encoder and decoder to California and I spent the next three weeks doing the work. Then, I FedEx’ed everything to Western Technologies.

Right about that time I got sick and was out of the running for a couple of weeks. Once I was up and around again I tackled some leftover problems with picture stability that the guys in California had been observing. ” After fixing it, it all worked remarkably well.

There was still however, one major problem that needed attention: The cost of the decoder circuitry was too high for a successful consumer product. TI couldn’t see moving forward unless we came up with a less expensive design.

A feasibility system that lowered the cost was functional by Christmas, and a meeting was set at TI for early the following January.

The demo to TI’s research group was “wildly successful” according to Jay. Representatives from Toys “R” Us were present too, and their reaction was also great. There was general speculation that video dealers could take one to two hundred thousand pieces by fall. Some excitement also centered on using the system on Cable TV, with cable operators producing the program material. I had always visualized that use of 4-C as a potential winner and designed the system with over-the-cable (or over-the-air) compatibility.

There was plenty of enthusiasm all around.

In February, TI completed a half-inch thick research report on a focus group that had played with a 4-C system. TI’s report on the focus group’s response to what they now referred to as their “TIM Interactive Video System” was also very favorable…however, some rain was about to fall on our parade.

It seemed that the bean counters at TI were worried about entering a new business area and were less than impressed with the cost and sales projections. Their trepidation caused the VP in charge of the project to get cold feet. Negative news soon began trickling out of TI. Sure enough, within a month the VP nixed further work and TIM was dead in the water, at least as far as TI was concerned.

The name of the game now was to find a new licensee…but fast, before technology moved past TIM and made it obsolete. As for me, my time was spoken for; Sanders and Magnavox were jointly pursuing Activision for infringement of the original video game patents. I spent that June and July in San Francisco, living in the Embarcadero Hyatt with assorted lawyers, and was in and out of Federal District Court for several weeks.

When we finally finished there (successfully, I might add) I flew down to L.A. and visited Jay at Western Technology and took a side-trip to Ken Curran’s lab. The TIM system was there and Ken gave me a demo…it worked like a charm!…but still no placement for the product.

• Fisher-Price to the rescue

Fisher-Price in upstate New York, decided to spring for a $20-50K advance to secure an option for F-P. In return, we committed ourselves to design and build a hand-held remote control for the TIM system.

Working on the R/C was a fun assignment and we soon had it working well. Meanwhile I also investigated ways to cut the cost of the system, once again. In the process, I found that there were some simple ways to make the “gap” audio noise go away. That impacted the cost in a very favorable direction.

Everything went well on the Fisher-Price project until the New York Toy Fair in February 1988. View Master showed up with a demo of an interactive, videotape-based system. It sounded like they were doing much of what our 4-C system did.

I wrangled an invitation to see and play with the system. It was set up in one of the many small rooms of the Toy Fair buildings. There wasn’t much to be seen except for a hand control and the TV set. The much ballyhooed View Master’s Interactive Video System – whatever that was – remained hidden from view behind a curtain. Naturally, I managed to peek behind the camouflage and saw a rat’s nest full of electronics hidden there. It was obvious that whatever it was, it had a long way to go before it could be called a product.

What’s more, it didn’t work well and it was boring. With nobody looking on, I pushed a button on the hand control to see what would happen. It promptly zapped the whole system; there were some red faces among the View Master crew behind the curtain, the demo was over and I left pronto before anybody started asking questions. My conclusion was: “Nothing to worry about!”

It was obvious that View Master had jumped the gun…they were nowhere near ready to display that lash-up at Toy Fair. Somebody in marketing, no doubt, had committed to a Show-and-Tell at the Fair, ready or not. I did not shed any tears over their problems.

Wouldn’t you know it, the negative fallout in the press about this poorly performing Interactive Video System was enough to spook Fisher-Price. They decided in their corporate wisdom to walk away from TIM and that was that!

Two outs and nobody on base!

That was the end of Four-Corners a.k.a. TIM; all that good work for nothing; what’s more, I hadn’t seen any of that $250K Jay had collected from TI, or the money collected from Fisher Price. Chalk up another one to experience!