OTA in Android - general knowledge

FOTA in Android (Mechanism, Implementation & Behind the scene)
-Firmware updation in Android over the air

What is FOTA?

Firmware Over-the-Air (FOTA) is a method used for update software on mobile phones and tablet computers etc.

FOTA in Android?

In android, FOTA can update current firmware, kernel, boot/system/recovery images, single/multiple application etc.

Major components involved in Android FOTA mechanism are:

•        FOTA Server with push notification support
•        OTA Updater application in Android device
•        Recovery console in Android device

FOTA procedure in Android?

In Android smartphones/tablets FOTA is performed in following way:

•         Device register to server for push notification of firmware/software updates.
•         Server sends the update notification to the device.
•         Device perform polling requests to server for new update.
•         User performs manual update check.
•         If update is available, get the update file URL from server.
•         Fetch update file and save in cache partition.
•         Verify the update file signature and checksum.
•         Reboot in recovery mode with update installation command.
•         Recovery console verifies the signature of update file.
•         Recovery console executed the updater script from update file.
•         Device Reboots after installation.

How to implement FOTA in Android (excluding push notification)

1. Create the Cryptographic signature for the OTA signing (required for releasing, not needed for demo)

There are four keys that can be used during development for authentication purposes:

Ø  testkey — a generic key for packages that do not otherwise specify a key.

Ø  platform — a key for packages that are part of the core platform.

Ø  shared — a  key for things that are shared in the home/contacts process.

Ø  media — a  key for packages that are part of the media/download system.

The default AOSP keys are found in the build/target/product/security directory.

(*.pk8 are the private keys, *.x509.pem are the corresponding certificate/public keys)

We can generate our keys with the shell script located at /development/tools/make_key

Procedure to create Keys is as follows:

• First remove the existing AOSP keys :

• $ rm build/target/product/security/*.p*

  view rawRemoveOldKeys.sh hosted with ❤ by GitHub
• Create New keys :

• 	$ /development/tools/make_key build/target/product/security/media ‘/C=US/ST=Utah/L=Salt Lake City/O=Android/OU=Android/CN=Android/emailAddress=android@example.com’
  	$ /development/tools/make_key build/target/product/security/platform ‘/C=US/ST=Utah/L=Salt Lake City/O=Android/OU=Android/CN=Android/emailAddress=android@example.com’
  	$ /development/tools/make_key build/target/product/security/shared ‘/C=US/ST=Utah/L=Salt Lake City/O=Android/OU=Android/CN=Android/emailAddress=android@example.com’
  	$ /development/tools/make_key build/target/product/security/testkey ‘/C=US/ST=Utah/L=Salt Lake City/O=Android/OU=Android/CN=Android/emailAddress=android@example.com’

  view rawCreateNewKeys.sh hosted with ❤ by GitHub
• Verify the created keys :

• 	$ build/target/product/security/openssl x509 -noout -subject -issuer -in media.x509.pem
  	$ build/target/product/security/openssl x509 -noout -subject -issuer -in shared.x509.pem
  	$ build/target/product/security/openssl x509 -noout -subject -issuer -in platform.x509.pem
  	$ build/target/product/security/openssl x509 -noout -subject -issuer -in testkey.x509.pem

  view rawVerifyKeys.sh hosted with ❤ by GitHub
• Re-sign the OTA file content after creation :

• $ /build/tools/releasetools/sign_target_files_apks -d build/target/product/security/ out/target/product/ product_name/someotafile.zip out/target/product/ product_name/signed-target_files.zip

  view rawRe-signOTAfile.sh hosted with ❤ by GitHub
• Re-sign the OTA file manually with SignApk.jar :

• $ java -jar /build/tools/signapk/signapk.jar testkey.x509.pem testkey.pk8 [update.zip] [update-signed.zip]

  view rawRe-signManuallyBySignApk.jar.sh hosted with ❤ by GitHub

      2.  If the new update package can have older “build date time” than only we should comment the following line from following file :
(otherwise skip this step):
/build/tools/releasetools/ota_from_target_files

      "script.AssertOlderBuild(ts);"

	def WriteFullOTAPackage(input_zip, output_zip):
	…
	# comment the following line:
	script.AssertOlderBuild(ts);

view rawota_from_target_files.py hosted with ❤ by GitHub

As the mentioned line creates the assert in the updater script for older build and aborts the installation if the update file is having older “build date time”.

 3. Create an FOTA update file with the command

$ make otapackage

view rawMakeOtapackage.sh hosted with ❤ by GitHub

      It will generate the full Android OS OTA update file in the out/target/product/product_name/someotafile.zip

 4. If we need to create any partial/incremental update then we can do with following command:

$ make target-files-package

view rawMaketarget-files-package.sh hosted with ❤ by GitHub

  It will only include the specified packages in OTA update file, its build logic is in build/core/Makefile.

 5. Now upload the zip on OTA server and whenever the update check for your device comes at server
After verifying device, rom values from the request provide the Rom info in the form of Jason String.

 6. Create an Android application with following permissions:

	<uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED" />
	<uses-permission android:name="android.permission.INTERNET" />
	<uses-permission android:name="android.permission.ACCESS_CACHE_FILESYSTEM" />
	<uses-permission android:name="android.permission.DELETE_CACHE_FILES" />
	<uses-permission android:name="android.permission.REBOOT" />

view rawAndroidMenifestPermissions.xml hosted with ❤ by GitHub

So that application can do following tasks:

• Perform polling request to server using internet on each time device boots/ manual update check by user.
• Update can be downloaded to /cache partition.
• For installation in recovery mode device can rebooted by the application.

  7. Create a Activity with intent filter "android.settings.SYSTEM_UPDATE_SETTINGS"
   This intent is fired from /packages/apps/Settings/res/xml/device_info_settings.xml

	<activity
	android:name=".FOTAUpdaterActivity"
	android:label="System Updates">
	<intent-filter>
	<action android:name="android.settings.SYSTEM_UPDATE_SETTINGS" />
	<action android:name="android.intent.action.MAIN" />
	</intent-filter>
	</activity>

view rawAndroidMenifest.xml hosted with ❤ by GitHub

So that when user navigates to System settings -> About device-> System Updates, our activity can be launched.

         8. Create an AsyncTask, which can be called whenever polling is required to be performed to check the update on server, the doInBackground() method will have pseudo code as follows: 

	private RomInfo doInBackground(Void... notused) {
	//Check Network availability
	ArrayList<BasicNameValuePair> params = new ArrayList<BasicNameValuePair>();
	params.add(new BasicNameValuePair("device",android.os.Build.DEVICE.toLowerCase()));
	params.add(new BasicNameValuePair("rom", Utils.getRomID()));
	HttpClient client = new DefaultHttpClient();
	HttpGet get = new HttpGet(SERVER_PULL_URL + "?" + URLEncodedUtils.format(params, "UTF-8"));
	HttpResponse r = client.execute(get);
	int status = r.getStatusLine().getStatusCode();
	HttpEntity e = r.getEntity();
	if (status == 200) {
	String data = EntityUtils.toString(e);
	JSONObject json = new JSONObject(data);
	if (json.has("error")) {
	error = json.getString("error");
	return null;
	}
	return new RomInfo(
	json.getString("rom"),
	json.getString("version"),
	json.getString("changelog"),
	json.getString("url"),
	json.getString("md5"),
	Utils.parseDate(json.getString("date")));
	}
	}

view rawRomCheckerAyncTask.java hosted with ❤ by GitHub

        9. Create a broadcast receiver with intent filter "android.intent.action.BOOT_COMPLETED"

	<receiver android:name="com.example.UpdateCheckReceiver">
	<intent-filter>
	<action android:name="android.intent.action.BOOT_COMPLETED"></action>
	</intent-filter>
	</receiver>

view rawAndroidMenifest.xml hosted with ❤ by GitHub

        So that when device boots up we can start our UpdateCheckReceiver class, which can Schedule a repeating alarm to send the pending intent to this same broadcastReceiver class to instantiate a new AsyncTask, to perform polling on server, to check the update.

  10. On a valid RomInfo Object received, Start the Downloader AsyncTask to download the update zip file from server, the doInBackground() method will have pseudo code as follows:

	private int doInBackground(Void... notused) {
	final File dest = new File("/cache/update.zip");
	InputStream is = null;
	OutputStream os = null;
	try {
	URL getUrl = new URL(romInfo.url);
	destFile.getAbsolutePath());
	URLConnection conn = getUrl.openConnection();
	conn.connect();
	is = new BufferedInputStream(conn.getInputStream());
	os = new FileOutputStream(destFile);
	byte[] buf = new byte[4096];
	int nRead = -1;
	int totalRead = 0;
	while ((nRead = is.read(buf)) != -1) {
	os.write(buf, 0, nRead);
	totalRead += nRead;
	}
	// Verify Md5 checksum here
	} catch (Exception e) {
	e.printStackTrace();
	destFile.delete();
	} finally {
	if (is != null) {
	try { is.close(); }
	catch (Exception e) { }
	}
	if (os != null) {
	try { os.flush(); os.close(); }
	catch (Exception e) { }
	}
	}
	return 0;
	}

view rawDownloaderAsyncTask.java hosted with ❤ by GitHub

  11. On successful download of the update zip file from server we must verify the cryptographic signature of the downloaded update file, and then we can install it as follows:

	try {
	File otafileloc = new File("/cache/update.zip");
	// Verify the cryptographic signature before installing it.
	RecoverySystem.verifyPackage(otafileloc, null, null);
	// Reboots the device into recovery mode to install the update package.
	RecoverySystem.installPackage(getApplicationContext(), otafileloc);
	} catch (IOException e) {
	e.printStackTrace();
	} catch (GeneralSecurityException e) {
	e.printStackTrace();
	}

view rawVerify&Install.java hosted with ❤ by GitHub

It will reboot the device in recovery mode, where recovery console will first verify the signature of the file then execute the updater script of that update zip file, which will install the updates i.e. (FOTA).
If any problem occurs it will show an android error symbol and wait for user reboot, otherwise It will reboot in to updated android after successful installation.

   Behind the scene

·        Releasetools (which creates OTA update file)

– It Create digitally signed software updates from a target-files-package (TFP)

-  TFP generated by Android* build system

–   We need to substitute testing for production keys inside a TFP

·        SW Update UI intent from AOSP Settings application

-  android.settings.SYSTEM_UPDATE_SETTINGS

-   Intent launched when user navigates to Settings->About device->System updates

·        android.os.RecoverySystem APIs

– Framework APIs to verify & install SW updates

–   Handles verification of OTA update digital signature

-   Also used to engage Factory Data Reset

-   RecoverySystem doesn’t write to BCB due to permissions on doing raw block device I/O

– Writes RC command files into /cache/recovery/command, and reboots into Recovery Console
– Typical RC commands which may be supplied in the /cache/recovery/command file:

       --send_intent=anystring - write the text out to recovery.intent
    --update_package=path - verify install an OTA package file
    --wipe_data - erase user data (and cache), then reboot
    --wipe_cache - wipe cache (but not user data), then reboot
    --set_encrypted_filesystem=on|off - enables / diasables encrypted fs
    --just_exit - do nothing; exit

·        /misc partition

–   Very small size, does not contain a filesystem

–   Tiny partition used for communication between Recovery Console (RC) and bootloader, and for RC to save state information

–   Contains Bootloader Control Block (BCB)

Ø  command[32]: Commands for the bootloader

• “boot-recovery” boot into RC instead of Android

• Other platform-specific commands may be implemented for update tasks that must be done by the bootloader

• If empty, garbage, or no known commands matched, normal Android boot

Ø  status[32]: Return status field written by bootloader after performing platform-specific commands

• No specification, platform-dependent

Ø  recovery[1024]: Command line for Recovery Console

• Arguments tokenized by ‘\n’

• Invalid if first argument not ‘recovery’

·        Recovery Console (RC) (Alternate boot environment)

– Comments at the beginning of bootable/recovery/recovery.c are out of date

-   A few ways to enter –

Ø  reboot recovery from a shell

Ø  RecoverySystem APIs in Android Framework

Ø  OEMs often implement a bootloader ‘magic key’

– Verify & Apply SW updates

– Perform Factory Data Reset

– Typically controlled by “/cache/recovery/command” file left by Android RecoverySystem APIs

-   Pictorial interface, no localization

-   Hidden non-localized menu for manual tasks

-   Other platform-specific tasks as implemented in Recovery Console UI plug-in

-   Log files saved in /cache/recovery

Ø  All stdout/stderr from RC and Updater

Ø  Edify ui_print() commands

-   Upon startup, looks for command line arguments in decreasing precedence:

Ø  Actual command line to ‘recovery’, debug-only scenario

Ø  BCB.recovery

-   Android framework RecoverySystem doesn’t write to BCB due to permissions on doing raw block device I/O, So Recovery console (RC) Always copies arguments into BCB.recovery and sets BCB.command to “boot-recovery”

Ø  Makes sure we keep booting into RC with the same arguments in event of unexpected power loss

Ø  Don’t rely solely on /cache/recovery/command for this

-   finish_recovery()

Ø  Called when requested operations (SW update, factory data reset, etc) are complete, whether successful or failed

Ø  BCB is cleared so that subsequent reboot goes back into Android

Ø  Copies all logs to /cache/recovery/

–   If no arguments were given to RC, displays error image and waits for menu input

·        Bootloader Integration

-   Linux kernel should write “boot-recovery” into BCB.command and zero out BCB.recovery if “recovery” is supplied as a reboot() argument from android framework

–   Implement in a driver via register_reboot_notifier()

-   Bootloader selects boot image (or other task) based on BCB.command

–   BCB.command is persistent; keep booting into RC until RC clears it

–   Garbage or zeroed out contents should simply boot into Android

·        Updater

–   SW update logic, binary inside SW update package

–   AOSP implementation runs script in Edify language

–   Platform-specific tasks implemented in plug-ins

·        updater script

-   Updater script written in Edify language

-   Created by python script in AOSP /build/tools/releasetools/edify_generator.py & /build/tools/releasetools/ota_from_target_files

-   Placed inside update zip file in /META-INF/…/updater-script