Live video through browser

Last week we delved into using SMS on a mobile phone to control a browser which is on TV. This browser could show canned media (or even live media) from the internet. What we didn't get into is displaying live video from a camera.

Flash Camera Viewing

Flash does a pretty good job of allowing us to view cameras that are connected to our local machines and allows us to see them in a browser so we can continue to utilize web technology to display dynamic graphics and text.

This snippet of ActionScript allows us to view a local live camera in a Flash Player:
// Camera or Video Input Capture
// http://www.adobe.com/livedocs/flash/9.0/ActionScriptLangRefV3/flash/media/Camera.html
var camera:Camera = Camera.getCamera();

// Set it's size and framerate
camera.setMode(640,480,30);

// Microphone or Line-in Capture
// http://www.adobe.com/livedocs/flash/9.0/ActionScriptLangRefV3/flash/media/Microphone.html
var microphone:Microphone = Microphone.getMicrophone(); 

// Video Display Component
// http://www.adobe.com/livedocs/flash/9.0/ActionScriptLangRefV3/flash/media/Video.html
var video:Video = new Video(); 

// Set Video Postion
video.x = 0;
video.y = 0;

// Set video size
video.height = 480;
video.width = 640;

// Display video component
addChild(video);

// Attach camera to video component
video.attachCamera(camera);
		
To use this code, you can use Flash 9 (or greater) or Flex (free for academic) or mxmlc (open source command line Flash compiler).

Assuming Flash, open up Flash, Create a new Flash File (ActionScript 3). On the "Timeline" highlight the first frame in the first layer and select "Actions" from the "Window" menu. You can paste the above code into that window and save.

After it is saved, you can go to "Control", "Test Movie" to try it out.

You should get a control panel that looks like this:



in which case you should select "allow".

You'll then see video from your default device. To change the source of the video, right click or cntrl-click on the video and select "settings", highlight the webcam icon on the bottom/far-right to access your other video input sources:



"USB Class Video" is most likely your built-in iSight (or an external USB webcam)
"IIDC Firewire" would be a firewire camera or webcam (original iSight or FireI)
"DV Video" would be a firewire connected DV Camera (probably the highest quality)


You can download the Flash Project (FLA) file for this: display_video.fla

Placing Flash on TV Browser

To put this Flash file inside the browser, "Publish" it and use the following embed code in your HTML:

		
	<object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=10,0,0,0" width="640" height="480" id="display_video" align="middle">
	<param name="allowScriptAccess" value="sameDomain" />
	<param name="allowFullScreen" value="false" />
	<param name="movie" value="display_video.swf" />
	<param name="quality" value="high" />
	<param name="bgcolor" value="#ffffff" />
	<param name="wmode" value="opaque" />	
	<embed src="display_video.swf" quality="high" bgcolor="#ffffff" width="640" height="480" name="display_video" align="middle" allowScriptAccess="sameDomain" allowFullScreen="false" type="application/x-shockwave-flash" pluginspage="http://www.adobe.com/go/getflashplayer" wmode="opaque" />
	</object>
		
Here is an example: display_video.html

Web + TV

Because we are using a browser to display what will eventually be on TV, it becomes easy to allow interaction from the audience via browser as well.

This type of scenario is generally called "Enhanced TV" or Two-Screen Interactive Television

More information:
ITVDictionary.com: 2 Screen Interactive TV
Wikipedia: Two-Screen Solutions
Wikipedia: Enhanced TV

Control Panel

First though, it is important that we develop a way to decide what will show up on the TV via some kind of control panel.

Since we are using AJAX to update our screen, we can use a separate browser which determines what should be fed to it. This seperate browser would in effect be a control panel.

Continuing with our example from last week of using SMS messages, let's build an additional component which allows a producer or director to select which messages should show up.

Here is our script which receives and logs the SMS messages from Textmarks:
		
<?
        // This PHP file: textmarks.php gets hit with phone and message when someone texts 41411 liveweb
        // http://itp.nyu.edu/~sve204/leit_fall11/textmarks.php?phone=\p&text=\0 

        // Phone Number:
        $phone = $_GET['phone'];

        // Message:
        $message = $_GET['text'] . "\n";

        // Save the message to the text file which changes my pages
        $fp=fopen('/home/sve204/public_html/leit_fall11/textmarks.txt','a');
        fwrite($fp,$message);
        fclose($fp);

        echo "Yay!!!  Thanks";
?>
We can leave this as is for now.

Here is the script which is hit by our "browser as TV" that pulls up messages from the text file that is being written to by the above script:
<?
	$contents = file_get_contents('/home/sve204/public_html/leit_fall11/textmarks.txt');

	$contents_array = split("\n",$contents);
	
	$last_message = $contents_array[sizeof($contents_array) - 1];
	
	echo $last_message;
?>
It pulls up the last message from the contents array.

If it is saved as get_last_message.php and pulled by AJAX on our TV browser it will always return the last message.

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
		<title>Display Video and Last Message</title>
		<script type="text/javascript" src="ajax.js"></script>
		<script type="text/javascript">

			// A variable to hold the interval id
			var interval = null;	
	
			// A function to call our server side file or script
			// This get's triggered by the interval (in function setup_ajax()
			function call_ajax()
			{
				makeHttpRequest('http://itp.nyu.edu/~sve204/leit_fall11/get_last_message.php',ajax_return);
			}

			// A function that gets called when the server responds
			// This is where you would do what you want with the response
			function ajax_return(response)
			{
				document.getElementById("textsection").innerHTML = response;
				document.getElementById("secondtextsection").innerHTML = response;

				interval = setTimeout("call_ajax()",2000);
			}
	
			// Setup AJAX function, creates a timeout so that we run something periodically
			function setup_ajax()
			{
				// Set interval of 2000 milliseconds
				// Keeps going over and over again
				interval = setTimeout("call_ajax()",2000);
			}			
		
			// Register setup_ajax with the onload event of the window (when it is done loading)..	
			// This makes setup_ajax run after the browser renders the page
			function addOnLoad()
			{
				if (window.addEventListener)
				{
					window.addEventListener('load', setup_ajax, false);
				}
				else if (window.attachEvent)
				{
					window.attachEvent('onload', setup_ajax);
				}
			}
			// Tell the browser to run the addOnLoad function above
			addOnLoad();
		</script>
		
		<style>
			#video {
				z-index: 0;
				color: black;
				position: absolute;
				top: 0px;
				left: 0px;
				width: 640px;
				height: 480px;
			}
			
			#textsection {
				z-index: 10;
				position: absolute;
				top: 320px;
				left: 0px;
				width: 640px;
				height: 160px;
				/*background-color: blue;*/
				color: white;
				text-align: center;
				font-size: 24pt;
			}
			
			#secondtextsection {
				z-index: 9;
				position: absolute;
				top: 322px;
				left: 0px;
				width: 640px;
				height: 158px;
				/*background-color: blue;*/
				color: black;
				text-align: center;
				font-size: 24pt;
			}
		</style>
	</head>	
	<body>
		<div id="video">
			<object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=10,0,0,0" width="640" height="480" id="display_video" align="middle">
			<param name="allowScriptAccess" value="sameDomain" />
			<param name="allowFullScreen" value="false" />
			<param name="movie" value="display_video.swf" /><param name="quality" value="high" /><param name="bgcolor" value="#ffffff" /><param name="wmode" value="opaque" />	<embed src="display_video.swf" quality="high" bgcolor="#ffffff" width="640" height="480" name="display_video" align="middle" allowScriptAccess="sameDomain" allowFullScreen="false" type="application/x-shockwave-flash" pluginspage="http://www.adobe.com/go/getflashplayer" wmode="opaque" />
			</object>
		</div>
		<div id="secondtextsection">Text Section</div>
		<div id="textsection">Text Section</div>	
	</body>
</html>
View it

This may not be exacly what you want. You may want to moderate the messages that come in or manually choose which to display.

To accomplish this, we can insert a script in the middle which takes messages from one text file and saves them into a "safe for display" file. This would be an HTML page in combination with a PHP script that does the work.

If we keep the HTML the same and simply change the text file that "get_last_message.php" reads from we can ensure that those messages are the ones we want to display:
<?
	$contents = file_get_contents('/home/sve204/public_html/leit_fall11/textmarks_good.txt');

	$contents_array = split("\n",$contents);
	
	$last_message = $contents_array[sizeof($contents_array) - 1];
	
	echo $last_message;
?>
Now we have to write a means for the moderator to approve those messages:
<?

	// If a "goodmessage" came in, write it to the textmarks_good.txt file
	if (isset($_GET['goodmessage']))
	{
        // Save the message to the text file which changes my pages
        $fp=fopen('/home/sve204/public_html/leit_fall11/textmarks_good.txt','w');
        fwrite($fp,$_GET['goodmessage']);
        fclose($fp);
	}
	
	// Get all of the messages
	$allmessages = file_get_contents('/home/sve204/public_html/leit_fall11/textmarks.txt');
	$messages_array = split("\n",$allmessages);
?>
<html>
	<head>
		<title>Moderator</title>
	</head>
	<body>
		Click on message to display:

<? // Loop through the messages, starting in reverse for ($i = sizeof($messages_array) - 1; $i >= 0; $i--) { if ($messages_array[$i] != "") { echo '<a href="moderator.php?goodmessage=' . $messages_array[$i] . '">' . $messages_array[$i] . '</a><br />' . "\n"; } } ?> </body> </html>
Try It

Command and Control

Taking it a step further, let's go through a generic framework for controlling what happens on the screen while taking input from "producers" or "viewers".

This package: live_moderator.zip includes what we have previously covered but in addition it uses a database.

The SQL to create the database is:
--
-- Table structure for table `tvcontrol`
--

CREATE TABLE IF NOT EXISTS `tvcontrol` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `command_content` text NOT NULL,
  `command_key` varchar(128) NOT NULL,
  `approved` tinyint(1) NOT NULL DEFAULT '0',
  `sent` tinyint(1) NOT NULL DEFAULT '0',
  `timecode` bigint(20) NOT NULL DEFAULT '0',
  PRIMARY KEY (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=91 ;		
		
The display_video_ajax_db.html file is the television output. It constantly queries get_approved_commands.php using AJAX to get any commands from the database that the "moderator" has approved and executes them.

The command_moderator.php file allows the moderator to submit and approve new commands.



display_video_ajax_db.html

Within this file, there is a "live" Flash video viewer. There is also a "textsection" displayed at the bottom.

The JavaScript in this file hits the "get_approved_commands.php" script and parses the response every 2 seconds. If the "command key" is "javascript" it executes that JavaScript allowing dynamic changes of the elements on the page.

If the "command key" is one of the id's of the elements on the page, it replaces the contents of that element with the "command contents".

command_moderator.php

This is a mixture of HTML and PHP. It queries the database, pulls up any commands that haven't been sent to the screen and haven't been approved. If you click on one of the commands, that submits it for approval.

Additionally, it allows you to enter a new command as per the rules in display_video_ajax_db.html.

For instance, you could enter "Hello There" as the "command" and "textsection" as the "key" and then approve. You would see on the display_video_ajax_db.html that the "textsection" changes to "Hello There".

Building an interface for the audience

If you wanted to allow the audience to contribute comments, the easiest thing to do would be to use the submission form within "command_moderator.php" as an example.

Here is a stub: (called audience.php)
<?
	// Database connect function
	$mySql = null;
	
	function sqlConnect() 
	{
		# Configuration Variables
		$hostname = "localhost";
		$dbname = "NETID";
		$username = "NETID";
		$password = "MYSQL-PASSWORD";
		
		$mySql = mysql_connect($hostname, $username, $password) or die (mysql_error());
		mysql_select_db($dbname, $mySql) or die(mysql_error());
		
		return $mySql;
	}

	if (isset($_POST['submit']))
	{
		// Connect to the database
		$mySql = sqlConnect();

		$query = "insert into tvcontrol (command_key, command_content)
					values ('" . mysql_escape_string($_POST['command_key']) . "',
							'" . mysql_escape_string($_POST['command_content']) . "');";
		mysql_query($query);
	}
?>
<html>
<body>
Hi Audience.  Enter a comment below:<br />
<form action="audience.php" method="POST">
	<input type="hidden" name="command_key" value="textsection" /><br />
	Have your say: <input type="text" name="command_content" /><br />
	<input type="submit" name="submit" value="submit" />
</form>		
</body>
</html>
		
Try it

Using this as a framework, you now have the ability to take any type of audience input and the ability to route it to the screen for display.

Go to town!