PHP Bitbucket Deployment Script

Posted in Web Development, on 4th of May, 2013 | ,

Only recently I introduced version control software into my workflow (GIT), I started using GitHub for public repositories, and Bitbucket later on for private repositories. Since researching methods of deploying scripts from Bitbucket to my servers I thought I would develop my own Bitbucket Deployment Script.

This article shows you how to create a Bitbucket Deployment Script and connect it as a POST service to your git repository, allowing file changes to be automatically updated on your server once a commit has been pushed to the remote repository.

Adding a POST service to your Bitbucket Repository

Access the repository you would like to deploy on Bitbuckets website, goto the settings page for that repository. Click on the services tab, choosing POST as the service. A text input should appear below allowing you to enter the URL of the bitbucket deployment script.

setup_bitbucket_service

Capturing the POST service on the Bitbucket deployment script

Now Bitbucket will send our prevously entered url details about the latest commit, we need to capture this information and modify the required files.

Alot of information is sent from bitbucket but we only need information about the commits, I have cut out the information that we dont need:

stdClass Object(
    [commits] => Array(
            [0] => stdClass Object(
                    [node] => b634f1bd6c29
                    [files] => Array(
                            [0] => stdClass Object(
                                    [type] => removed
                                    [file] => test/6.php
                                )
                        )
                    [message] => add file 1
                )
        )
)

Now that we know what how the information looks, we can capture it from the $_POST variable payload by creating a file that will be accessed via bitbuckets POST service that we setup in the previous section:

// capture information sent from bitbucket
$json = isset($_POST['payload']) ? $_POST['payload'] : false;
 
// stop if no payload has been submitted
if(!$json)
	return false;
 
$data = json_decode($json);
 
// process all commits
if(count($data->commits) > 0){
	foreach($data->commits as $commit){
		$node = $commit->node;		// get node string
		$files = $commit->files;	// get array of file changes
		$message = $commit->message;	// get commit message 
 
		// process file changes here
	}
}

Applying the repositories changes

The code should do the following:

  1. Download the current repository node
  2. Extract the downloaded zip to the working repository
  3. Merge existing files with changes from the Bitbuckets commit list
  4. Cleanup directory, remove zip file and extracted files

Bitbucket Deployment Script – Final Code:

<?php 
/**
 * Bitbucket POST Deployment Class
 * @author  James Collings <james@jclabs.co.uk>
 * @version 0.0.1
 */
class BitbucketDeploy{
 
	private $user = '';	// Bitbucket username
	private $pass = '';	// Bitbucket password 
	private $repo = 'deploy-test';	// repository name
	private $deploy = './deploy-test/';	// directory deploy repository
	private $download_name = 'download.zip'; // name of downloaded zip file
	private $debug = true;	// false = hide output
	private $process = 'update'; // deploy or update
 
	// files to ignore in directory
	private $ignore_files = array('README.md', '.gitignore', '.', '..');
 
	// default array of files to be committed
	private $files = array('modified' => array(), 'added' => array(), 'removed' => array());
 
	function __construct(){
 
		$json = isset($_POST['payload']) ? $_POST['payload'] : false;
		if($json){
			$data = json_decode($json);	// decode json into php object
 
			// process all commits
			if(count($data->commits) > 0){
				foreach($data->commits as $commit){
 
					$node = $commit->node;	// capture repo node
					$files = $commit->files;	// capture repo file changes
					$message = $commit->message;	// capture repo message
 
					// reset files list
					$this->files = array(
						'modified' => array(), 
						'added' => array(), 
						'removed' => array());
 
					foreach($files as $file){
						$this->files[$file->type][] = $file->file;
					}
 
					// download repo
					if(!$this->get_repo($node)){
						$this->log('Download of Repo Failed');
						return;
					}
 
					// unzip repo download
					if(!$this->unzip_repo()){
						$this->log('Unzip Failed');
						return;
					}
 
					// append changes to destination
					$this->parse_changes($node, $message);
 
					// delete zip file
					unlink($this->download_name);
				}
			}else{
				// if no commits have been posted, deploy latest node
				$this->process = 'deploy';
 
				// download repo
				if(!$this->get_repo('master')){
					$this->log('Download of Repo Failed');
					return;
				}
 
				// unzip repo download
				if(!$this->unzip_repo()){
					$this->log('Unzip Failed');
					return;
				}
 
				$node = $this->get_node_from_dir();
				$message = 'Bitbucket post failed, complete deploy';
				if(!$node){
					$this->log('Node could not be set, no unziped repo');
					return;	
				}
 
				// append changes to destination
				$this->parse_changes($node, $message);
 
				// delete zip file
				unlink($this->download_name);
			}
		}else{
			// no $_POST['payload']
			$this->log('No Payload');
		}
	}
 
	/**
	 * Extract the downloaded repo
	 * @return boolean
	 */
	function unzip_repo(){
		// init zip archive helper
		$zip = new ZipArchive;
 
		$res = $zip->open($this->download_name);
		if ($res === TRUE) {
			// extract files to base directory
		    $zip->extractTo('./');
		    $zip->close();
		    return true;
		}
		return false;
	}
 
	/**
	 * Download the repository from bitbucket
	 * @param  string $node 
	 * @return boolean
	 */
	function get_repo($node = ''){
 
		// create the zip folder
		$fp = fopen($this->download_name, 'w');
 
		// set download url of repository for the relating node
		$ch = curl_init("https://bitbucket.org/$this->user/$this->repo/get/$node.zip");
 
		// http authentication to access the download
		curl_setopt($ch, CURLOPT_USERPWD, "$this->user:$this->pass");
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
		curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
 
		// disable ssl verification if your server doesn't have it
		curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
		curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
 
		// save the transfered zip folder
		curl_setopt($ch, CURLOPT_FILE, $fp);
 
		// run the curl command
		$result = curl_exec($ch);	//returns true / false
 
		// close curl and file functions
		curl_close($ch);
		fclose($fp);
		return $result;
	}
 
	/**
	 * Apply the repository changes add, edit, delete
	 * @param  string $node    
	 * @param  string $message 
	 * @return void
	 */
	function parse_changes($node = '', $message = ''){
		$src = "./$this->user-$this->repo-$node/";
 
		if(!is_dir($this->deploy))
			$this->process = 'deploy';
 
		$this->log('Process: '.$this->process);
		$this->log('Commit Message: '.$message);
 
		$dest = $this->deploy;
		$real_src = realpath($src);
 
		if(!is_dir($real_src)){
			$this->log('Unable to read directory');			
			return;
		}
 
		$output = array();
 
		$objects = new RecursiveIteratorIterator(
			new RecursiveDirectoryIterator($real_src), 
			RecursiveIteratorIterator::SELF_FIRST);
 
		foreach($objects as $name => $object){
 
			// check to see if file is in ignore list
			if(in_array($object->getBasename(), $this->ignore_files))
				continue;
 
			// remove the first '/' if there is one
			$tmp_name = str_replace($real_src, '', $name);
			if($tmp_name[0] == '/')
				$tmp_name = substr($tmp_name,1);
 
			switch($this->process){
				case 'update':
					// only update changed files
					if(in_array($tmp_name, $this->files['added'])){
						$this->add_file($src . $tmp_name, $dest . $tmp_name);
					}
					if(in_array($tmp_name, $this->files['modified'])){
						$this->modify_file($src . $tmp_name, $dest . $tmp_name);
					}
				break;
				case 'deploy':
					$this->add_file($src . $tmp_name, $dest . $tmp_name);
				break;
			}
		}
 
		// delete all files marked for deleting
		if(!empty($this->files['removed'])){
			foreach($this->files['removed'] as $f){
				$this->removed($dest . $f);
			}
		}
 
		$this->delete($src);
	}
 
	/**
	 * Delete folder recursivly
	 * @param  string $path 
	 * @return void
	 */
	private function delete($path) {
	    $objects = new RecursiveIteratorIterator(
	    	new RecursiveDirectoryIterator($path), 
	    	RecursiveIteratorIterator::CHILD_FIRST);
 
	    foreach ($objects as $object) {
	        if (in_array($object->getBasename(), array('.', '..'))) {
	            continue;
	        } elseif ($object->isDir()) {
	            rmdir($object->getPathname());
	        } elseif ($object->isFile() || $object->isLink()) {
	            unlink($object->getPathname());
	        }
	    }
	    rmdir($path);
	}
 
	/**
	 * Retrieve node from extracted folder name
	 * @return string
	 */
	private function get_node_from_dir(){
		$files = scandir('./');
		foreach($files as $f){
			if(is_dir($f)){
				// check to see if it starts with 
				$starts_with = "$this->user-$this->repo-";
				if(strpos($f, $starts_with) !== false){
					return substr($f, strlen($starts_with));
				}  
			}
		}
		return false;
	}
 
	/**
	 * Write log file
	 * @param  string $message 
	 * @return void
	 */
	private function log($message = ''){
		if(!$this->debug)
			return false;
 
		$message = date('d-m-Y H:i:s') . ' : ' . $message . "\n";
		file_put_contents('./log.txt', $message, FILE_APPEND);
	}
 
	/**
	 * Add new file
	 * @param string $src  
	 * @param string $dest 
	 * @return  void
	 */
	private function add_file($src, $dest){
		$this->log('add_file src: '. $src . ' => '.$dest);
		if(!is_dir(dirname($dest)))
			@mkdir(dirname($dest), 0755, true);
		@copy($src, $dest);
	}
 
	/**
	 * Replace file with new copy
	 * @param  string $src  
	 * @param  string $dest 
	 * @return void
	 */
	private function modify_file($src, $dest){
		$this->log('modify_file src: '. $src . ' => '.$dest);
		@copy($src, $dest);
	}
 
	/**
	 * Delete file from directory
	 * @param  string $file
	 * @return void
	 */
	private function removed($file){
		$this->log('remove_file file: '. $file);
 
		if(is_file($file))
			@unlink($file);
	}
 
}
new BitbucketDeploy();
?>

Download the current repository

Bitbucket stores all nodes of its repositories in zip folders, they can be accessed with curl, since we are using a private repository, we need to authenticate curl to access the zip file by passing the CURLOPT_USERPWD and grab the with CURLOPT_FILE.

function get_repo($node = ''){
 
	// create the zip folder
	$fp = fopen($this->download_name, 'w');
 
	// set download url of repository for the relating node
	$ch = curl_init("https://bitbucket.org/$this->user/$this->repo/get/$node.zip");
 
	// http authentication to access the download
	curl_setopt($ch, CURLOPT_USERPWD, "$this->user:$this->pass");
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
	curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
 
	// disable ssl verification if your server doesn't have it
	curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
	curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
 
	// save the transfered zip folder
	curl_setopt($ch, CURLOPT_FILE, $fp);
 
	// run the curl command
	$result = curl_exec($ch);	//returns true / false
 
	// close curl and file functions
	curl_close($ch);
	fclose($fp);
	return $result;
}

Extracting the Zip Archive

First we need to open the zip file using zip archive open function, then we extract the contents to the current directory using the zip archive extractTo function:

function unzip_repo(){
	$zip = new ZipArchive;
	$res = $zip->open($this->download_name);
	if ($res === TRUE) {
	    $zip->extractTo('./');
	    $zip->close();
	    return true;
	}
	return false;
}

Merging Files

To merge the committed changes, we have to loop through all the downloaded files and compare them to the list of files needing changed. The core of this process takes place in the parse_changes() function.

$objects = new RecursiveIteratorIterator(
	new RecursiveDirectoryIterator($real_src), 
	RecursiveIteratorIterator::SELF_FIRST);
 
foreach($objects as $name => $object){
 
	// check to see if file is in ignore list
	if(in_array($object->getBasename(), $this->ignore_files))
		continue;
 
	// remove the first '/' if there is one
	$tmp_name = str_replace($real_src, '', $name);
	if($tmp_name[0] == '/')
		$tmp_name = substr($tmp_name,1);
 
	switch($this->process){
		case 'update':
			// only update changed files
			if(in_array($tmp_name, $this->files['added'])){
				$this->add_file($src . $tmp_name, $dest . $tmp_name);
			}
			if(in_array($tmp_name, $this->files['modified'])){
				$this->modify_file($src . $tmp_name, $dest . $tmp_name);
			}
		break;
		case 'deploy':
			$this->add_file($src . $tmp_name, $dest . $tmp_name);
		break;
	}
}
 
// delete all files marked for deleting
if(!empty($this->files['removed'])){
	foreach($this->files['removed'] as $f){
		$this->removed($dest . $f);
	}
}

Cleanup

No we have merged the new files, we need to delete the extracted repository and downloaded zip files. The following function loops through the directory, deleting all files then removing the directory.

private function delete($path) {
    $objects = new RecursiveIteratorIterator(
    	new RecursiveDirectoryIterator($path), 
    	RecursiveIteratorIterator::CHILD_FIRST);
 
    foreach ($objects as $object) {
        if (in_array($object->getBasename(), array('.', '..'))) {
            continue;
        } elseif ($object->isDir()) {
            rmdir($object->getPathname());
        } elseif ($object->isFile() || $object->isLink()) {
            unlink($object->getPathname());
        }
    }
    rmdir($path);
}

All that is left to do is delete the zip file with php unlink() function.

  • Syamil

    James, this is very useful, thank you! Would you consider turning this into a WP plugin. Could be an extremely useful tool for themes/plugins deployment :)

    Cheers!

    • jcollings

      Cheers Syamil , never though of creating it as a wordpress plugin, but it does sounds like it could be quite useful.

      Stay tuned for the plugin development progress.

      • Syamil

        Awesome! I’d be happy to help out if you need any for the project :)

  • therobbrennan

    Nicely done, man. This was exactly what I was hoping to learn how to do. Thanks for sharing!

  • teamcrisis

    Any way to do this without exposing U & P? Possibly with keys instead?

    • jcollings

      @teamcrisis:disqus the reason there is a username and password required is to authenticate the download of the private repository. If you are wanting to access a public repo then you don’t to authenticate to download it.

      If you want to access a private repository without the the username and password, you can setup git on the server and download the repo using a deployment key.

      • teamcrisis

        @jcollings:disqus thank you for the explanation and clarification! Any word on the WP plugin?

        • jcollings

          a WP Plugin is under development currently, i am currently figuring out a secure way to 2 way encrypt the bitbucket password. The latest development version of the plugin can be found at https://github.com/jcollings/jc-bitbucket-deployment

          • teamcrisis

            Awesome, thank you! I’d be glad to test and help out in any way I can

          • teamcrisis

            @jcollings:disqus any progress made on the encryption or plugin in general?

          • jcollings

            The plugin is coming along, just haven’t had much time recently to update it.

          • jcollings

            Storage of your account password has now been encrypted and salted depending on you having the php mcrypt library installed. The latest version of JC-Deploy can be found http://jamescollings.co.uk/wordpress-plugins/ along with a quick setup guide.

  • christian_giupponi

    That’s exactly what I need but I can’t find the “Service” tab in my Bitbucket repo!
    How can I do that?

    • jcollings

      Bitbucket have recently updated there interface, the services tab has changed to “Hooks” , once on that settings page you need to choose POST from the hooks dropdown

      • christian_giupponi

        Thanks for your answer, I have add a new hook but nothing appened after a new pull…
        This is my vars in yout code:

        private $user = ‘username’; // Bitbucket username
        private $pass = ‘password’; // Bitbucket password
        private $repo = ‘repo’; // repository name
        private $deploy = ‘./’; // directory deploy repository
        private $download_name = ‘download.zip’; // name of downloaded zip file
        private $debug = true; // false = hide output
        private $process = ‘update’; // deploy or update

        in $deploy I have set the root of my server.
        What I miss?

        • teamcrisis

          Did you get your issue resolved? I’m having similar issues.

          • christian_giupponi

            No luck!

        • jcollings

          Are any files being created by the deploy script?
          Do they have enough permissions to be able to create the files?

          Try putting this code in the section of the __construct() function just below ‘//No post['payload'] to force it to deploy.


          // if no commits have been posted, deploy latest node
          $this->process = 'deploy';

          // download repo
          if(!$this->get_repo('master')){
          $this->log('Download of Repo Failed');
          return;
          }

          // unzip repo download
          if(!$this->unzip_repo()){
          $this->log('Unzip Failed');
          return;
          }

          $node = $this->get_node_from_dir();
          $message = 'Bitbucket post failed, complete deploy';
          if(!$node){
          $this->log('Node could not be set, no unziped repo');
          return;
          }

          // append changes to destination
          $this->parse_changes($node, $message);

          // delete zip file
          unlink($this->download_name);

          Then open the deploy script from your browser and it should deploy, otherwise check what is happening in the log.txt file.