cURLing Android Market stats in my website

Last week I thought it would be nice to collect some statistics about my apps in the Android Market. Seeing Websites like androlib.com and androidpit.de I thought it shouldn’t be a problem. However, apart from strazzere.com I haven’t found much useful information. Since I’m only interested in the stats of my own apps I took a deeper look on Google’s Developer Console for the Android Market.

I played a bit with Firebug and learned that the Developer Console is a GWT application, that JSON is used to get the app descriptions from the server, and that the GWT stuff is horrible to reverse engineer. Luckily I found a post that shows how to get the stats from the Android Marketplace with PHP/cURL. It didn’t worked for me at first but after toying around a bit it works now for me. However, I still have no clue what the JSON stuff I get from the GWT server means and I’m only guessing the most important values. It will likely break when I change anything in my developer account.

Below is the PHP script I use to fetch the data from the developer console. 99% is copied from Craige Thomas (I have absolutely no clue about PHP or cURL!). I only added the part that guesses the position of the values and the caching. The script is used to produce the output of the widget on the right.

60)) {

	//do google authorization

	$data = array('accountType' => 'GOOGLE',
	'Email' => 'YOUR GOOGLE LOGIN'
	'Passwd' => 'YOUR PASSWORD',
	'source'=>'',
	'service'=>'androiddeveloper');

	$ch = curl_init();
	curl_setopt($ch, CURLOPT_URL, "https://www.google.com/accounts/ClientLogin");
	curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
	curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
	curl_setopt($ch, CURLOPT_POST, true);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
	curl_setopt($ch, CURLOPT_POSTFIELDS, $data);

	$output = curl_exec($ch);

	$info = curl_getinfo($ch);
	curl_close($ch);

	//grab the AUTH token for later

	$auth = '';
	if($info['http_code'] == 200) {
		preg_match('/Auth=(.*)/', $output, $matches);

		if(isset($matches[1])) {
			$auth = $matches[1];
		}
	}

	//login to Android Market
	//this results in a 302
	//I think this is necessary for a cookie to be set

	$ch = curl_init ("http://market.android.com/publish?auth=$auth");
	curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt');
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
	$output = curl_exec($ch);

	//go to the Developer Console
	$ch = curl_init ("http://market.android.com/publish/Home");
	curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
	curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt');
	$output = curl_exec($ch);

	//grab the JSON data
	$perm = "746E1BE622B08CBF950F619C16FCFF1E";
	$headers = array(
		"Host: market.android.com",
		"User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2",
		"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
		"Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3",
		"Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7",
		"Keep-Alive: 115",
		"Connection: keep-alive",
		"Content-Type: text/x-gwt-rpc; charset=utf-8",
		"X-GWT-Permutation: $perm",
		"X-GWT-Module-Base: http://market.android.com/publish/gwt/",
		"Referer: http://market.android.com/publish/gwt/$perm.cache.html");

	//not sure what x-gwt-permutation means, I think it may have to do with which version of GWT they serve based on your browser

//Change here?
	$postdata = "5|0|4|http://market.android.com/publish/gwt/|14E1D06A04411C8FE46E62317C1AF191|com.google.wireless.android.vending.developer.shared.AppEditorService|getFullAssetInfosForUser|1|2|3|4|0|";

	$ch = curl_init ("http://market.android.com/publish/editapp");
	curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
	curl_setopt($ch, CURLOPT_POST, 1);
	curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
	curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt');
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

	$output = curl_exec($ch);
	
	$output = substr($output,4);

	$json = json_decode($output);
	$csv = explode(',',$output);

	$apps = array();

	$index = 0;
	$app_count = 0;
	for($i = 0; $i < sizeof($csv); ++$i) {
		if (is_array($json[$i])) {
			$innerArray=$json[$i];
			break;
		}
		if (strpos($csv[$i], ".") !== false) {
			if ($index==1) $apps[$app_count][comments]=$csv[$i];
			else if ($index==2) $apps[$app_count][rating]=$csv[$i];
			else if ($index==4) $apps[$app_count][installs]=$csv[$i];
			else if ($index==6) $apps[$app_count][total]=$csv[$i];
			$index++;
			if ($index==7) {
				$index=0;
				$app_count++;
			}
		}
	}
	for($i = 4; $i < sizeof($innerArray); ++$i) {
		if ((substr($innerArray[$i],-1)=='k') && (substr($innerArray[$i-2],0,17)=='GetImage?imageId=')){
			$app_count--;
			$apps[$app_count][icon]="http://market.android.com/publish/".$innerArray[$i-2];
			$apps[$app_count][name]=$innerArray[$i-3];
			$apps[$app_count][size]=$innerArray[$i];
			$apps[$app_count][package]=$innerArray[$i-1];
		}
	}

	$Handle = fopen('market_stats.txt', 'w');
	fwrite($Handle, '');
	for($i = 0; $i < sizeof($apps); $i++) {
		fwrite($Handle, "");
	}
	fwrite($Handle, '
"); fwrite($Handle, ''); fwrite($Handle, ""); fwrite($Handle, ''); fwrite($Handle, $apps[$i][name].'
'); fwrite($Handle, 'Total installs '.round($apps[$i][total])); fwrite($Handle, "
'); fclose($Handle); $Handle = fopen('market_stats_history.txt', 'a'); for($i = 0; $i < sizeof($apps); $i++) { fwrite($Handle, $apps[$i][package].', '); fwrite($Handle, round($apps[$i][total]).', '); fwrite($Handle, time()); fwrite($Handle, "\n"); } fclose($Handle); } $readHandle = fopen('market_stats.txt', 'r'); echo fread($readHandle, filesize('market_stats.txt')); fclose($readHandle); ?>

10 thoughts on “cURLing Android Market stats in my website

  1. Niels Post author

    Unfortunately there is a problem with the code. Once in a while Google changes “something”. Afterwards, my script does not work anymore. The suspicious line is
    $postdata = “5|0|4|http://market.android.com/publish/gwt/|8962DA36A8613A353E193A7B0C216D42|com.google.wireless.android.vending.developer.shared.AppEditorService|getFullAssetInfosForUser|1|2|3|4|0|”;

    There is a magic number included and I have no idea how I can get this number using PHP. I use the Firefox plug-in Firebug to look at the “editapp” request. Then I just copy the post data and it works again. If you know how to fix it I wouldn’t mind to learn about it :-)

  2. Brad

    Great script, working for me.

    I’m going to modify it to timestamp each value and then have it do the math to return just the # of downloads that day vs. total downloads.

    If you’re interested in my changes, let me know.

  3. Michelle

    Great script, working for me.

    I’m going to modify it to timestamp each value and then have it do the math to return just the # of downloads that day vs. total downloads.

    If you’re interested in my changes, let me know.

  4. Niels Post author

    Hi Michelle & Brad,

    I would be interested in any improvement :-) The problem with the code is the line below “//Change here?”. I have to update the magic number (in the code it is “14E1D06A04411C8FE46E62317C1AF191″) every now and then because something changed in the developer console. Unfortunately I have no idea where to get this number from.

    Niels

Comments are closed.