November 7, 2001

MPEG4IP Project
===============

The MPEG4IP project provides a standards-based system for encoding, 
streaming, and playing MPEG-4 encoded audio and video. To achieve 
this we've integrated a number of existing open source packages, and 
also created some original code to fill in the gaps.

Please note this project is intended for developers who are interested
in MPEG-4 audio and video, and Internet streaming. It is not intended
for end-users. Please read all the legal information!

Also note that the primary development focus of this project is the 
Linux platform. If you're going to use the package on other platforms,
especially non-UNIX platforms, you'll probably have some work to do.

Please use the SourceForge site to report problems, suggest enhancements,
ask questions, etc. The URL is http://www.sourceforge.net/projects/mpeg4ip

There is also a project web site at http://www.mpeg4ip.net/ that has some
general information on MPEG4IP.   


Upgrading
=========

Note: If you have used a pre-0.9 version of mpeg4ip please read this section.

In version 0.9, the mp4 file format library has been reimplemented. The 
new implementation addresses a number of bugs and shortcomings of the old 
library. Also it has been extensively tested for conformance with the mp4
standard and interoperability with mp4 files generated by other packages.
That's the good news, the bad news is that this change-over has some impact
on users of the MPEG4IP package. 

A number of the utilities that used the old library have been deprecated.
Specifically:

MP4 UTILITIES
    DEPRECATED                    REPLACEMENT
    server/util/mp4apkt            server/mp4creator
    server/util/mp4vpkt            server/mp4creator
    server/util/mp4dump            lib/mp4v2/util/mp4dump
    server/util/mp4extract         lib/mp4v2/util/mp4extract

The file mpeg4ip/doc/encoding/encoding.htm describes the usage of mp4creator.
There is also a man page for it.

Note: you may wish to uninstall mp4apkt and mp4vpkt. 
The install process will not do this automatically.

The MPEG4IP player, gmp4player, will by default use the new mp4 library.
This can cause old MPEG4IP mp4 files to fail to play due to compliance 
problems in file format. The player has an option under the Help/Debug menu,
"Use old mp4 library" that can be used to switch to the old mp4 library 
and play these old files. However, we recommend that you regenerate these 
mp4 files so they will be compatible with the mp4 standard. 

In most cases, old mp4 files can be regenerated using the mp4v1regenerate
script which doesn't require re-encoding hence it's much faster than 
regenerating from the original raw content. The usage is simply:

$ mp4v1regenerate foo.mp4 (the new file will be foo_new.mp4)


Overview
========

There are now two ways to use MPEG4IP to create content. 

The older method assumes that you've somehow managed to capture raw audio
and/or video into a file. That's the starting point from which you can use
the MPEG4IP encoding tools to create an MP4 file. The simplest method being 
to use the 'mp4encode' script. Detailed instructions for this script and the
individual tools that it uses are in doc/encoding/encoding.htm  

The new method is an integrated live encoding tool called mp4live.  This 
program is designed to make it easy to create MP4 files or transmit live
audio/video streams over the network. It can even do both things 
simultaneously!  The key requirement to use this tool is to have a video 
capture device and a Video for Linux (v4l) driver for it. So far we've 
tested with the bttv driver for Brooktree based video capture cards, and 
the qce driver for Logitech QuickCam Express USB webcams. Both of these 
solutions can be acquired for about $50 US! Please see the file 
mpeg4ip/server/live/README for more information about mp4live.

Once an MP4 file is prepared, it can be placed in the content 
directory of a streaming server. We include Apple's Darwin Streaming 
Server with MPEG4IP, but any server that understands MP4 files (or
hinted Quicktime files) can be used. 

When the content is encoded and available on the server, you can run
the player. Start 'gmp4player' and then enter the RTSP URL to the server 
and the content, (command line works too). E.g.  
    $ gmp4player rtsp://myserver.mydomain.com/mycontent.mp4

Although we're focussed on streaming, the player will also playback from
a local file. E.g
    $ gmp4player mycontent.mp4

That's not all! The player is not limited to local playback of MP4 files.
It can also read AVI, CMP, DIVX, AAC, MP3, and WAV. This is useful for 
debugging since the encoded data can be check independently of the MP4 
file container, and known good content, such as your favorite MP3, can be 
used to verify that the player is working correctly with your hardware. E.g.
    $ gmp4player mymusic.mp3

And that's still not all... You can set up your player to run a playlist
by creating a simple text file with the extension .gmp4_playlist, with
each item (file or stream) on it's own line.

Note: If you prefer a no UI version of the player, or your system does not
include GTK or GLIB, just the bare video window, 'mp4player' is available 
to fit that need.


Legal
=====

As you might imagine, integrating a significant number of open source 
packages creates a complicated legal situation. Here's the plain language 
overview. For the complete story, you'll have to wade through the licenses.

For the original code that we've written, it's under the Mozilla Public
License 1.1, included in the file 'COPYING' or from http://www.mozilla.org/mpl
 
Each third party package has it's own license (it's quite a menagerie). 
Before you do anything, you should take a look at each package. They are 
listed in the next section.

Like most modern codecs, MPEG-4 Video and Audio codecs are almost certainly 
subject to patent royalities. The problem is that you wouldn't know whom to
pay. For MPEG-2 this problem was addressed by establishing a patent pool that
charges a relatively modest fee for products to license the patents in the 
pool. This is admistered by MPEG-LA, http://www.mpeg-la.org/  For MPEG-4,
the MPEG-4 Industry Forum is looking at doing roughly the same thing.
See http://www.m4if.org for details. In any case, you need to consider this 
issue before you start building derivative works. We are not warranting or 
indemnifying you in any way for patent royalities! YOU ARE SOLELY RESPONSIBLE 
FOR YOUR OWN ACTIONS!


Redistributed Open Source Packages
==================================

This is the list of redistributed open source packages that are included 
in MPEG4IP:

Name:       Apple Darwin Streaming Server
Version:    3.0.1
Origin:     http://www.apple.com/projects/streaming
Author:     Apple Computer
Directory:  mpeg4ip/server/DSS
License:    Apple Public Source License (APSL)
Notes:      The package includes the APSL version 1.1, however Apple 
            release version 1.2 of the APSL on January 4, 2001. This
            version of the license can be used instead if desired.

Name:       avilib
Version:    1999
Origin:     http://www.munich-vision.de/lavtools
Author:     Rainer Johanni
Directory:  mpeg4ip/lib/avi
License:    GNU Library General Public License (LGPL), version 2 1991
Notes:      Extracted from lavtools package,
            Used only by the avi2raw utility

Name:       ffmpeg
Version:    0.3.4
Origin:     http://www.sourceforge.net/projects/ffmpeg
Author:     Gerard Lantau
Directory:  mpeg4ip/server/video/ffmpeg
License:    GNU General Public License (GPL), version 2 1991
Notes:      Code redistributed with MPEG4IP is the video encoder library
            portion of the ffmpeg project. See the ffmpeg site to obtain
            the complete distribution.

Name:       Freeware Advanced Audio Coder (FAAC) / Decoder (FAAD)
Version:    1.1
Origin:     http://www.sourceforge.net/projects/FAAC
Author:     Menno Bakker
Directory:  mpeg4ip/server/audio/faac
            mpeg4ip/player/lib/audio/faad
License:    GNU Library General Public License (LGPL), version 2 1991 
            ISO MPEG-4 reference software license for original base code
Notes:      Use may require payment of patent royalties

Name:       ISO MPEG-4 reference software
Version:    2000
Origin:     http://www.iso.ch/ittf
Author:     ISO
Directory:  mpeg4ip/common/video/mpeg4
License:    Other, see source files
Notes:      Use may require payment of patent royalties
            Many contributors are listed in the individual source files.

Name:       LAME
Version:    3.70
Origin:     http://www.mp3dev.org/mp3/
Authors:    Mike Cheng, Mark Taylor     
Directory:  mpeg4ip/server/audio/lame
License:    GNU Library General Public License (LGPL), version 2 1991 
            for modifications
            ISO MPEG-2 reference software license for orignial base code
Notes:      Use may require payment of patent royalties
            Many contributors are listed in the individual source files.

Name:       Open DivX
Version:    4.0, alpha 50 for encoder, alpha48 for decoder 
Origin:     http://www.projectmayo.com/
Author:     Project Mayo
Directory:  mpeg4ip/server/video/divx, and mpeg4ip/player/lib/video/divx
License:    DivX Open license, version 2.1
Notes:      DivX license can be found in above directories, as
            divx_open_license.txt
            Note this license has requirements on content generated
            by the encoder when the content is used for commerical purposes.
            Many contributors are listed in the individual source files.

Name:       Quicktime for Linux
Version:    1.2
Origin:     http://heroine.linuxave.net/quicktime.html 
Author:     Adam Williams
Directory:  mpeg4ip/lib/mp4
License:    Other, see README_ORIGINAL
Notes:      This directory also contains original MPEG4IP code.

Name:       Simple DirectMedia Layer
Version:    1.2.2
Origin:     http://www.libsdl.org/
Author:     Sam Lantinga
Directory:  mpeg4ip/lib/SDL
License:    GNU Library General Public License (LGPL), version 2 1991

Name:       sndfile
Version:    0.0.16
Origin:     http://www.zip.com.au/~erikd/libsndfile
Author:     Erik de Castro Lopo
Directory:  mpeg4ip/server/lib/sndfile
License:    GNU Lesser General Public License, version 2.1 1999
Notes:      Used by FAAC

Name:       splay
Version:    0.9.4
Origin:     http://splay.sourceforge.net/
Author:     Woo-jae Jung
Directory:  mpeg4ip/player/audio/mp3
License:    GNU Library General Public License (LGPL), version 2 1991

Name:       UCL Common Multimedia Library
Version:    1.2.8
Origin:     http://www-mice.cs.ucl.ac.uk/multimedia/software/common/
Author:     University College London
Directory:  mpeg4ip/lib/rtp
License:    Other, see COPYRIGHT
Notes:      This directory contains additional third-party code with 
            different licenses.


We will attempt to keep up with the current stable releases of
these packages. 

If you have questions or problems that are specific to one of 
these packages, please look at the package for how best to contact
the authors/maintainers. 


Building and Installing
=======================

This package uses GNU autoconf, automake, and libtool. It attempts to 
follow the GNU conventions for open source packages. This is complicated
by the fact that we build on many other packages. Where the package was 
already using the GNU tools, we left things alone. Where the package was 
using it's own Makefile, we left things alone if it was a complicated 
Makefile. If it was straightforward we replaced it with an equivalent 
automake file.

If you have to install any of these tools (for example, automake is
not included with Mac OS X), find out where the others are installed
(which autoconf). If the path does not start with /usr/local, use the
"configure --prefix=<path before /bin on other packages>"

For example, if autoconf is in /usr/bin, use the "configure --prefix=/usr"
command when installing automake or libtools.

Note: libtool is frequently not installed on Linux systems (autoconf, 
and automake generally are). You can download libtool from a GNU mirror
site, or http://download.sourceforge.net/mpegip/libtool-1.3.5.tar.gz

In order to compile mpeg4ip, libtool must have a minimum version of 1.3.5.

In general, the code should be portable, but as someone once said 
"There's no such thing as portable code, just code that has been ported." 
When you find problems please be sure to use the SourceForge site to tell 
us what you encountered, and hopefully how you fixed it.

To build:
    ./bootstrap
    make
    make install (optional, typically need root privileges)

The bootstrap script will pass any arguments to the configure scripts.

Two options of potential interest are "--disable-server" and "--disable-player"
which disable the building of the server and player respectively. By
default both server and player are built.

If you are building on a system with an Intel x86 CPU clone, you may
need to specify the configuration option "--enable-mmx=no". The
configuration script automatically detects an x86 target CPU and 
enables MMX assembly code in the build (if the NASM assembler is available).
If your CPU doesn't support MMX instructions you will want to disable 
this feature. 

For the curious, the bootstrap script invokes the configure scripts of 
the included packages that have them, and then our own top level configure 
script is generated and run. At the end of this process all the Makefile's
are ready, and setup in the correct hierarchy. If this doesn't work for you,
you're free to hack as needed ;-)  

Please note that the Apple Darwin Quicktime Streaming Server is NOT part of
the current build process. If you want to build (and install) it you need to
perform that step separately. 

To build the Darwin Streaming Server:

    cd server/DSS
    ./Buildit

To install the Darwin Streaming Server from the build:
    cd server/DSS
    mkdir dist
    ./DSS_MakeRoot -f dist
    cd dist
    ./Install    (need root privileges)

See the documentation that accompanies the server on how to configure it
for your environment. 

Note the default content directory is /usr/local/movies.

There is a short "home video" sample file in mpeg4ip/samples that can 
be used to check that the build and install have succeeded.  First, 
try opening the file with gmp4player. If that works, then try copying 
the file to the streaming server's content directory (e.g. /usr/local/movies),
and enter the appropriate RTSP URL in gmp4player.

Note that the clip is encoded to minimize the size of the distribution, and
does not reflect the best quality that can be achieved.

You can find other mp4 clips at the Philips MPEG-4 site,
http://www.mpeg-4player.com/  Note that these clips are not hinted, so
they can't be streamed, only local playback is possible.

OS Supported
============

Currently, we have compiled and tested on the following platforms:
linux, freeBSD, BSD/OS, Mac OS X and windows.  For all varieties
of *nux, X11 is required.

To date we've built on Red Hat Linux 6.1, 6.2, 7.0, and 7.1 with
the native compilers, and on 7.1 with gcc 3.0.  We've built on 
freeBSD 4.3, and BSD/OS.

For BSD/OS, you may need a semaphore shim.  Contact Steven Schultz
for help on that (contact information in Authors).

When building on Mac OSX, you may get an error that libtoolize is
not installed correctly.  Mac OSX comes with libtool installed as
glibtool.  Automake (which doesn't come included with OSX) does not
correct its scripts.

To fix this problem, do a link of libtoolize to glibtoolize (ie:
ln -s libtoolize glibtoolize where ever glibtoolize is installed).

When building on Slackware-8.0, you need to consider the following 
(from maersk):
  The Slackware distribution does not as standard come with shared 
  libraries for libXv and libXxf86dga. In other cases, where you 
  have upgraded from xfree86 version 4.0 to 4.1, you may not have 
  compiled the shared versions. The fix to this is in general is this:

  # pushd /usr/X11R6/lib
  # rm -f libXv.so libXxf86dga.so
  # ld --whole-archive -shared -o \
      libXv.so libXv.a
  # ld --whole-archive -shared -o \
      libXxf86dga.so libXxf86dga.a
  # popd

  Note that you need to be root to do this.



Executables
===========

If you ran 'make install' with the defaults, then all the MPEG4IP executables
will end up in /usr/local/bin. 

For encoding tools, you get:

mp4live     Integrated, live encoding to file or network - Linux only

mp4encode   Front-end script to the following encoding tools:

avi2raw     Extracts raw audio/video tracks from an AVI file
lboxcrop    Vertically crops raw video to a new aspect ratio
divxenc     Encodes raw video into MPEG-4 encoded video using the Divx codec
faac        Encodes raw audio into MPEG-4 AAC encoded audio
lame        Encodes raw audio into MP3 encoded audio
mp4venc     Encodes raw video into MPEG-4 encoded video using ISO codec
mp4creator  Creates and hints audio/video tracks to an mp4 file

These are described in more detail in doc/encoding/encoding.htm


A few debugging tools are also included:

mp4extract  Utility to extract tracks from an MP4 file
mp4dump     Utility to dump MP4 file meta-information in text form
yuvdump     Utility to display a raw video file on the screen


For playback, you get:

gmp4player  Simple graphical interface player
mp4player   Bare video window with sync'ed audio


If you installed the Darwin Streaming Server, those executables will end
up in /usr/local/sbin.

DarwinStreamingServer    Provides streaming service for MP4 files
PlaylistBroadcaster      Provides multicast playlist service for MP4 files



Directory Structure
===================

If you're going to start hacking, a map of the territory may prove useful:

mpeg4ip - top level project directory

    config - autoconf files

    doc - the minimal doc we've written so far
        encoding - how to encode contenet
        ietf - copies of the relevant RFC's
        mcast - how to multicast 

    include - project wide includes 

    common - shared code
        video 
            mpeg4 - MPEG-4 video encoder/decoder

    lib - project wide libraries
        SDL - Simple DirectMedia Layer
        avi - AVI file format
        bitstream - MPEG style low level bitstream utlity
        config_file - Configuration file utility
        gnu
            getopt - gnu getopt routines
        mp4 - MP4 (aka MOV/Quicktime) file format library
        mp4v2 - new MP4 library written from scratch
            util - contains new mp4dump and mp4extract utilities
        msg_queue - SDL based Inter-thread messages utlity
        rtp - UCL RTP 
        sdp - Our own SDP
        win32 - libary files need for MS Windows

    player - player specific code
        lib - libraries specifically for the player
            audio
                faad - FAAD AAC decoder
                mp3 - MP3 decoder
            libhttp - Our own http client
            rtsp - Our own RTSP client 
            video
                divx - OpenDivx decoder
        src - the player executable
            codec - wrappers to interface codecs with the player
                aa - wrapper around FAAD
                divx - wrapper around OpenDivx
                mp3 - wrapper around MP3
                mpeg4 - wrapper around mpeg4v
                wav - wrapper around PCM wav
            osx - Mac OS X UI

    server - server specific code
        DSS - Darwin QuickTime Streaming Server
            <see server/DSS/Documenation/AboutTheSource.html>
        audio
            faac - AAC encoder program
            lame - MP3 encoder program
        lib - libraries specifically for the server tools
            sndfile - sound file formats, used by faac
        mp4creator - create and hint A/V tracks to an mp4 file
        mp4live - mp4 live encoding interface
            gui - gtk gui for mp4 live.
        util
            avi2raw - extract raw A/V data from AVI files
            lboxcrop - vertically crop raw video 
            mp4apkt - DEPRECATED MP4 audio packetizer
            mp4dump - DEPRECATED dump MP4 file meta-info contents
            mp4encode - front-end script to simplify encoding process
            mp4extract - DEPRECATED extract A/V data from MP4 files
            mp4vpkt - DEPRECATED MP4 video packetizer
            mp4vhdr - DEPRECATED MPEG-4 video header generator
            rgb2yuv - rgb to yuv converter
        video
            divx - OpenDivx encoder library and program
            ffmpeg - ffmpeg video encoder library

    util - generally useful utilities
        yuv - simple tools for examining raw video


Standards Compliance
====================

We're not only supporters of open source, we're supporters of open standards!
We've attempted to use the publically defined standards as much as possible.
Here's what we believe we are following. If you find something non-compliant,
please let us know. We certainly will want to fix it.

Here are the citations:

ISO/IEC 14496-1:2001 MPEG-4 Systems (includes MP4 file format)
ISO/IEC 14496-2:2000 MPEG-4 Video
ISO/IEC 14496-3:1999 MPEG-4 Audio (includes AAC)
ISO/IEC 11172-3:1993 MPEG-1 Audio (includes MP3)
ISO/IEC 13818-3:1998 MPEG-2 Audio (includes extensions to MP3)
ISO/IEC 13818-7:1997 MPEG-2 AAC

The ISO/IEC documents must be purchased from either ISO (www.iso.ch)
or one of the national bodies. In the US, ANSI is the representative
body, and provides an online store under www.ansi.org


IETF RFC 1889 & 1890 RTP

IETF RFC 2326 RTSP

IETF RFC 2327 SDP

IETF RFC 2250 RTP Payload for MPEG-1/2
    Note: we're just using the audio part for MP3.

IETF RFC 3016 RTP Payload for MPEG-4 Audio/Visual
    Note: we're implementing the MPEG-4 video part

IETF draft-vandermeer-mpeg4-simple-01.txt - work in progress
    Note: we're implementing the AAC audio part

The IETF RFCs can be found in mpeg4ip/doc/ietf


Note the MP4 file format is derived from Apple's QuickTime file format.
That specification is:

Apple Computer QuickTime File Format, June 28 2000
http://developer.apple.com/quicktime/


We're also involved in the Internet Streaming Media Alliance (ISMA) 
which seeks to standardize the protocols and formats used for streaming.
We believe MPEG4IP is interoperable with that organization's 1.0 Technical 
Specification. For more information see http://www.isma.tv


Known Problems
==============

* If you're playing through a NAT box, you may have to specify the
  default client IP ports.  Use the command RtpIpPortMin=<port> and
  RtpIpPortMax=<port> in the .gmp4player_rc file created in your home
  directory.  The IETF recommends a range of 6970 to 6999.

* The ISO MPEG-4 video codec bitrate control feature is broken. We've
  heard that a fix is available, so we're trying to track that down.
  Note that the OpenDivx encoder has bitrate control that works.

* The player for windows is sketchy at best.  We have included project
  files that should build.  We have noticed the following problems:
  - Only debug version seems to work.  
  - Release version seems to have problems with exceptions.  Our bytestream
    class will throw an exception when it has a problem - missing sequence
    numbers because of dropped packets, for example.  The release version,
    even though all projects have exception processing enabled, seems to
    crash when they occur.

=== END OF README ===
