Monday, September 26, 2011

The J2ME record management system

   The J2ME record management system (RMS) provides a mechanism through which MIDlets can persistently store data and retrieve it later. In a record-oriented approach, J2ME RMS comprises multiple record stores. An overview of J2ME RMS and MIDlet interfacing is given in Figure 1.

Figure 1. Overview of J2ME RMS and MIDlet interfacing
Overview of J2ME RMS and MIDlet interfacing
Each record store can be visualized as a collection of records, which will remain persistent across multiple invocations of the MIDlet. The device platform is responsible for making its best effort to maintain the integrity of the MIDlet's record stores throughout the normal use of the platform, including reboots, battery changes, etc.
A record store is created in platform-dependent locations, like nonvolatile device memory, which are not directly exposed to the MIDlets. The RMS classes call into the platform-specific native code that uses the standard OS data manager functions to perform the actual database operations.
Record store implementations ensure that all individual record store operations are atomic, synchronous, and serialized, so no corruption of data will occur with multiple accesses. The record store is timestamped to denote the last time it was modified. The record store also maintains a version, which is an integer that is incremented for each operation that modifies the contents of the record store. Versions and timestamps are useful for synchronization purposes.
When a MIDlet uses multiple threads to access a record store, it is the MIDlet's responsibility to coordinate this access; if it fails to do so, unintended consequences may result. Similarly, if a platform performs a synchronization of a record store with multiple threads trying to access the record store simultaneously, it is the platform's responsibility to enforce exclusive access to the record store between the MIDlet and its synchronization engine.
Each record in a record store is an array of bytes and has a unique integer identifier.
Managing the device database
The javax.microedition.rms.RecordStore class represents a RMS record store. It provides several methods to manage as well as insert, update, and delete records in a record store.
Managing record stores
To open a record store, the openRecordStore() method of javax.microedition.rms.RecordStore is invoked. public static RecordStore openRecordStore(String recordStoreName, boolean createIfNecessary) opens a record store with the given name recordStoreName. If there is no record store of that name, invoking this method creates one.
If the record store is already open, this method returns a reference to the same record store object.

Listing 1. Open a RecordStore
RecordStore rs = RecordStore.openRecordStore("MyAppointments",true);

Once all operations are done, a call to closeRecordStore() closes the record store with the given name. When a record store is closed, it is disabled for further operations.

Listing 2. Close a RecordStore

A named record store can be deleted by invoking the deleteRecordStore() method.

Listing 3. Delete a RecordStore

Inserting records
The MIDlet invokes the addRecord() method of javax.microedition.rms.RecordStore class to insert a new record into the record store. This is a blocking atomic operation and returns the recordId for the new record. The record is written to persistent storage before the method returns.
public int addRecord(byte[] data, int offset, int numBytes) inserts a record represented by an array of bytes data with offset as its starting index and numBytes as its length.

Listing 4. Insert a record
String appt = "new record";
byte bytes[] = appt.getBytes();

Updating records
Updating a particular record involves getting a handle for that record and setting new information.
public int getRecord(int recordId, byte[] buffer, int offset) returns the data stored in the given record in the byte array represented by buffer. public byte[] getRecord(int recorded) returns a copy of the data represented by recordId. public void setRecord(int recordId, byte[] newData, int offset, int numBytes) sets new information, a stream of bytes (newData) with offset as its starting index and numBytes as its length, at the record location represented by recordId.

Listing 5. Update a record
String newappt = "update record";
Byte data = newappt.getBytes();
Rs.setRecord(1, data, 0, data.length());

Deleting records
The MIDlet invokes the deleteRecord() method to delete a record from the record store.
public void deleteRecord(int recordId) deletes the record represented by recordId. The recordId for this record is not subsequently reused.

Listing 6. Delete a record

Data interpretation
The J2ME API provides certain interfaces to interpret the data stored in a record store. This process involves comparing records to determine their relative sort order. It also involves the filtering of contents depending on given conditions.
Comparing records
The MIDlet implements the RecordComparator interface and defines a compare (byte[] rec1, byte[] rec2) method to compare two candidate records. The return value of this method must indicate the ordering of the two records.

Listing 7. Comparing records and determine relative sort order
Int compare (byte[] b1, byte[] b2)
String s1 = new String(b1);
String s2 = new String(b2);

If (s1.compareTo(s2) > 0)
Return RecordComparator.FOLLOWS;
Else if (s1.compareTo(s2) == 0)
Return RecordComparator.EQUIVALENT;
Return RecordComparator.PRECEDES;

Enumerating records
The RecordEnumeration interface is responsible for enumerating records in a record store. It logically maintains a sequence of the recordIds of the records in a record store. The enumerator will iterate over all of the records (or only a subset, if an optional record filter has been supplied) in an order determined by a record comparator. If neither the filter nor the comparator are specified, the enumeration will traverse all records in the record store in an undefined order.

Listing 8. Enumerating records
RecordEnumeration re = rs.enumerateRecords(null, null, false);
If (re.hasNextElement())
Byte nextRec[] = re.nextRecord();

Filtering records
The MIDlet implements the RecordFilter interface, defining a filter that examines a record to see if it meets application-defined criteria. The application implements the RecordFilter's match() method to select the records to be returned by the RecordEnumeration.

Listing 9. Filtering records
Public boolean matches(byte[] candidate)
String s1 = new String(candidate);
If (s1.equals("XX"))
Returns true;
Returns false;

Developing a phone appointment diary
In this section, we'll demonstrate the functioning of J2ME RMS by building a phone appointment diary. This application will allow users to set appointments at certain dates and times, cancel appointments, or view a list of appointments already set. A quick screen view is shown in Figure 2.

Figure 2. The phone appointment diary at work
 The phone appointment diary at work
A complete list of user interface elements which comprise the various screens and screen elements of this application is available in the MID Profile API documentation that accompanies the J2ME Wireless Toolkit; for more detail on these elements, check out an earlier article I wrote for developerWorks (see the Resources section below for links to both).
A record store can store a record as a stream of bytes. In our application, the date and time entered by the user are concatenated into a string, converted into bytes, and stored.

Listing 10. Add a new appointment to the database
Public boolean matches(byte[] candidate)
String appt = apptName + " " + apptTime;
byte bytes[] = appt.getBytes();

Similarly, the application retrieves the record as a stream of bytes and converts it into a string. The string is in a ####AAAA format, where # denotes numbers representing time information and AAAA denotes characters representing a description of the appointment. The application parses the string to get the date and time information and displays them in a user-desired format, such as description - mm/dd/yyyy hh:mm AM_PM.

Listing 11. Retrieve a record from the record store
     byte b[] = rs.getRecord(j);
String str = new String(b,0,b.length);

Listing 12. Parsing data obtained from record store and displaying in a user-desired format
  if (Character.isDigit(str.charAt(i)))
at += str.charAt(i);
name += str.charAt(i);
time = Long.parseLong(at);
java.util.Date date = new java.util.Date(time);
java.util.Calendar rightNow = java.util.Calendar.getInstance();
String year = String.valueOf
String month = String.valueOf
(rightNow.get(java.util.Calendar.MONTH) + 1);
String day = String.valueOf

String displayName = name
+ "-" + year + " " + day;

The user is allowed to select certain appointments and delete them from the record store. Since the deleted recordId is not reused so as to maintain the original sequence in records, the record is marked as invalid by a distinctive string pattern.

Listing 13. Mark a record as deleted
String deactive = "@";
byte b[] = deactive.getBytes();
rs.setRecord(m+1, b,0,b.length);

As the application displays a list of appointments, it detects the string pattern of those invalid records and skips them.

Listing 14. Skip invalid records
if (!(str.startsWith("@")))
// Record is valid
// Record is invalid.

One of the important aspects of this application is the user interface. The various screens are as follows:
  • Welcome form: The welcome form displays a list of appointments already set and notifies the user if no appointments have been set. It provides various options to continue or exit the application.
  • Menu form: The menu form provides the user with the options to view an appointment, set a new appointment, or cancel an appointment, among others.
  • Display form: The display form displays the list of appointments already set.
  • Set form: The set form provides a date selection field and an input text field to provide the details of a new appointment. When the user chooses to save, the information is stored in the database.
  • Delete form: The delete form lists the set of appointments and provides the option to select one or more. If the user chooses to delete, the set of selected appointments is marked as invalid in the record store.
The application implements CommandListener and ItemStateListener interfaces that allow the application to respond to various events. ItemStateListener allows the application to receive events that indicate changes in the internal state of:
  • DateField, an editable component for presenting date and time
  • TextField, an editable text component
  • ChoiceGroup, a group of selectable elements

Listing 15. Acquire values from the screen
// The date value is set to a variable when the
// DateField item is changed
if (item == apptDate)
date = apptDate.getDate();
apptTime = String.valueOf(date.getTime());
// The name of appointment is set to a variable
//when the name input field is changed
if (item == apptField)
apptName = apptField.getString();

// If the ChoiceGroup item state on Delete form is
//changed, it sets an array of appointments selected for deletion
if (item == cg)

Listing 16 contains the complete listing for the sample application. See my previous article on J2ME in the Resources section for instructions on downloading a device emulator that will allow you to run this program on your desktop.
The bottom line
In this article, we covered the ability of MID applications to persistently store and retrieve data; the mechanism is modeled after a simple record-oriented database. The J2ME API javax.microedition.rms package provides a developers treasure chest of methods and interfaces to utilize this unique feature of MID application. You should now be able to integrate data storage into your own micro Java applications.

Tuesday, September 20, 2011

App Store Tip

HTTP Live Streaming in iOS

HTTP Live Streaming is a streaming standard proposed by Apple. See draft 5.
Files involved are
  • .m4a for audio (if you want a stream of audio only).
  • .ts for video. This is a MPEG-2 transport, usually with a h.264/AAC payload. It contains 10 seconds of video and it is created by splitting your original video file, or by converting live video.
  • .m3u8 for the playlist. This is a UTF-8 version of the WinAmp format.
Even when it's called live streaming, usually there is a delay of one minute or so during which the video is converted, the ts and m3u8 files written, and your client refresh the m3u8 file.
All these files are static files on your server. But in live events, more .ts files are added, and the m3u8 file is updated.
Since you tagged this question iOS it is relevant to mention related App Store rules:
  • You can only use progressive download for videos smaller than 10 minutes or 5 MB every 5 minutes. Otherwise you must use HTTP Live Streaming.
  • If you use HTTP Live Streaming you must provide at least one stream at 64 Kbps or lower bandwidth (the low-bandwidth stream may be audio-only or audio with a still image).


Get the streaming tools

To download the HTTP Live Streaming Tools do this:
Command line tools installed:


Descriptions from the man page:
  • Media Stream Segmenter: Create segments from MPEG-2 Transport streams for HTTP Live Streaming.
  • Media File Segmenter: Create segments for HTTP Live Streaming from media files.
  • Variant Playlist Creator: Create playlist for stream switching from HTTP Live streaming segments created by mediafilesegmenter.
  • Media Stream Validator: Validates HTTP Live Streaming streams and servers.
  • ID3 Tag Generator: Create ID3 tags.

Create the video

Install Macports, go to the terminal and sudo port install ffmpeg. Then convert the video to transport stream (.ts) using this FFMpeg script:

# bitrate, width, and height, you may want to change this
# strip off the file extension
=$(echo ${input} | sed 's/\..*//' )
# works for most videos
-y -i ${input} -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s ${WIDTH}x${HEIGHT} -vcodec libx264 -b ${BR} -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 7 -trellis 0 -refs 0 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate ${BR} -bufsize ${BR} -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 30 -qmax 51 -qdiff 4 -level 30 -aspect ${WIDTH}:${HEIGHT} -g 30 -async 2 ${output}-iphone.ts

This will generate one .ts file. Now we need to split the files in segments and create a playlist containing all those files. We can use Apple's mediafilesegmenter for this:
mediafilesegmenter -t 10 myvideo-iphone.ts
This will generate one .ts file for each 10 seconds of the video plus a .m3u8 file pointing to all of them.

Setup a web server

To play a .m3u8 on iOS we point to the file with mobile safari. Of course, first we need to put them on a web server. For Safari (or other player) to recognize the ts files, we need to add its MIME types. In Apache:

 AddType application/x-mpegURL m3u8
AddType video/MP2T ts

In lighttpd:

 mimetype.assign = ( ".m3u8" => "application/x-mpegURL", ".ts" => "video/MP2T" )

To link this from a web page:

<meta name="viewport" content="width=320; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;"/>
<video width="320" height="240" src="stream.m3u8" />