How To Install and Configure AppScale on Ubuntu 12.04

Introduction

AppScale is an open source computing platform designed to deploy Google App Engine applications on public clouds, private clouds, and on-premise clusters. AppScale is fully compatible with the Google App Engine APIs and has support for Python, Go, PHP, and Java. With AppScale you can migrate existing apps to any cloud compute platform, including DigitalOcean. Below you will find a list of the open source components used to serve a given API.

• Datastore API: Cassandra and ZooKeeper
• Memcache API: memcached
• Task Queue API: RabbitMQ and Celery
• XMPP API: ejabberd
• Channel API: strophe.js and ejabberd
• Blobstore API: Cassandra and ZooKeeper
• Images API: PIL
• Cron API: Vixie Cron

Prerequisites

For this tutorial, you will need:

• 4GB+ Droplet with Ubuntu 12.04.5

AppScale requires at least 2 GB of RAM to compile the required components in addition to the 2 GB of RAM that AppScale uses at idle. A minimum of 4 GB of RAM is strongly recommended for standard application deployments. It may be possible to use a 2 GB Droplet with a swap file. However, that is beyond the scope of this tutorial.

At the time of writing, AppScale only has official support for Ubuntu 12.04. If you modify the build script, it may be possible to install on Ubuntu 14.04. However, that is also outside the scope of this tutorial and may not be supported by the community.

The first 2 steps, installing AppScale and the AppScale Tools, must be run as the root user. The remaining steps can be run as a non-root user.

Step 1 — Install AppScale

For the first two sections we will want to run all the commands as the root user. If you are connected to the server as a sudo user, enter the root shell with:

sudo su

First, update your apt-get package index:

apt-get update

We are now ready to install AppScale. We will be compiling AppScale from source. Please note compiling source code can be very time consuming. Expect this process to take upwards of 15 minutes or longer to complete.

Ensure you are in the /root directory:

cd /root

Install Git so you can use it to download the AppScale source code:

apt-get install -y git-core

Clone the AppScale source code from GitHub:

git clone git://github.com/AppScale/appscale.git

Change to the appscale/debian directory, and run the build script.

cd appscale/debian
bash appscale_build.sh

Step 2 — Install AppScale Tools

The AppScale Tools are used to manage AppScale Clusters and deploy applications. These tools can be installed on a local machine or your server. For simplicity we will be installing the tools on our server. The install process on Mac OS X and Windows is very similar. You need to use Cygwin on Windows. See the GitHub Page for more information.

Change back to the /root directory:

cd /root

Clone the AppScale Tools source code from GitHub:

git clone git://github.com/AppScale/appscale-tools.git

Change to the appscale-tools/debian directory, and run the build script.

cd appscale-tools/debian
bash appscale_build.sh

After the build script completes, it would be a good idea to reboot.

reboot

Step 3 — Configure Your AppScale Deployment

For the remaining portion of this tutorial, you can run the AppScale Tools as any user. This does not need to be a sudo user. However, you will need to know the root user’s password when starting AppScale for the first time. AppScale will automatically create authentication certificates, and the root password will no longer be required when working with the AppScale Tools in the future.

After the server has finished rebooting, and you have established your SSH connection, you need to configure your AppScale deployment. The AppScale Tools require a configuration file every time you run the toolset. In this step we will create the configuration file called AppScalefile, start AppScale, and configure the administrator account.

Ensure you are in your user’s home directory:

cd ~

Create the initial AppScalefile configuration file:

appscale init cluster

Now, we will add the server’s IP address to the AppScalefile.

Open the file with nano:

nano AppScalefile

At the top of the file, you will see the following section:

# The deployment strategy (roles -> machines) that should be used in this
# AppScale deployment.
# The following is a sample layout for running everything on one machine:
ips_layout :
  master : your_server_ip
  appengine : your_server_ip
  database : your_server_ip
  zookeeper : your_server_ip

Replace the default IP address with your server’s IP address. When you are done editing the file, press CTRL-X, press Y to save, and press ENTER to overwrite the existing file name.

Now we can start AppScale from the same directory as the AppScalefile just created:

appscale up

AppScale will ask you to verify the host fingerprint and root password.

The authenticity of host '111.111.111.111 (111.111.111.111)' can't be established.
ECDSA key fingerprint is ab:3a:f0:87:c8:4e:8c:ba:59:0e:06:64:1b:f6:fe:e8.
Are you sure you want to continue connecting (yes/no)? yes

Type yes, and press ENTER. You will then see the following:

root@111.111.111.111's password: 

Enter the root user password, and press ENTER.

After entering the correct root password, you will see the following:

Generated a new SSH key for this deployment at /root/.appscale/appscale69de89364b624a8a9be1b7f45ac23d40
Starting AppScale 2.3.1 over a virtualized cluster.
Log in to your head node: ssh -i /root/.appscale/appscale69de89364b624a8a9be1b7f45ac23d40.key root@111.111.111.111
Head node successfully initialized at 111.111.111.111. It is now starting up cassandra.
Copying over deployment credentials
Starting AppController at 111.111.111.111
Please wait for the AppController to finish pre-processing tasks.

Please wait for AppScale to prepare your machines for use.
AppController just started

When starting AppScale, it may seem like it is hung at AppController just started. This is normal. It can take some time for all the AppScale components to initialize.

Eventually, you will see the following:

UserAppServer is at 111.111.111.111
Enter your desired admin e-mail address:

Create an admin user account. Enter the email address for the user, and provide a password. Remember these details. You will need them to access the AppScale Administration Panel.

After you create the admin user account, you will see:

Creating new user account admin@example.com
Creating new user account admin@example.com
Your XMPP username is admin@111.111.111.111
Granting admin privileges to admin@example.com
AppScale successfully started!
View status information about your AppScale deployment at http://111.111.111.111:1080/status

AppScale will provide you with a link to the administration panel. Usually in the following format. Typically the http address will automatically redirect to the secure https address.

http://your_server_ip:1080/status
https://your_server_ip:1443/status

Step 4 — The AppScale Administration Panel

Open the AppScale Administration Panel in your browser. The link should have been provided to you after starting AppScale:

http://your_server_ip:1080/status

You may be prompted to accept the self signed certificate.

From the AppScale Administration Panel, users can create their own accounts by clicking Create Account. However, you will need to change their permissions using the admin account before they can upload and remove their own apps.

Click the Login button in the top right. The Login button may look different on smaller screens, but it will still be green.

Login with the admin email and password you set in the previous step. You will then be presented with the AppScale status page.

The Administration Panel gives you access to server statistics and application statistics. You can also deploy and remove applications. It is fairly straight forward to deploy an application from the Administration Panel. For the purposes of this tutorial we will learn how to deploy an application from the command line. When you are finished exploring the Administration Panel, continue to the next step.

Step 5 — Deploying Your First Application

AppScale provides a collection of sample applications that are ready to deploy. These applications are a good way to test your AppScale cluster. They also familiarize you with the application deployment process.

You should be using the same user account, and your current directory should contain the AppScaleFile. This file contains all the configurations AppScale requires to manage you deployment.

Make sure we are back in your user’s home directory:

cd ~

From GitHub, clone the sample application source code to create the Guestbook App:

git clone https://github.com/AppScale/sample-apps.git

You will see the following as the source code downloads:

Cloning into 'sample-apps'...
remote: Counting objects: 15742, done.
remote: Total 15742 (delta 0), reused 0 (delta 0), pack-reused 15742
Receiving objects: 100% (15742/15742), 318.96 MiB | 23.52 MiB/s, done.
Resolving deltas: 100% (4944/4944), done.

The Guestbook App is a great way to test the datastore and authentication APIs.

Deploy the application:

appscale deploy sample-apps/go/go-guestbook/

You will be asked to assign an email address to your application. Enter the email address, and press Enter.

Enter your desired e-mail address: admin@example.com

This can be any email address. If the user does not already exist in the database, you will be prompted to set a password. For the purposes of this tutorial we decided to use the admin account.

Next, you will see the following:

Uploading initial version of app guestbookgo
We have reserved guestbookgo for your app
Tarring application
Copying over application
Please wait for your app to start serving.
Waiting 1 second(s) to check on application...
Waiting 2 second(s) to check on application...
Waiting 4 second(s) to check on application...
Waiting 8 second(s) to check on application...
Waiting 16 second(s) to check on application...
Your app can be reached at the following URL: http://111.111.111.111:8080

Open the URL provided in your browser, and you will be served by the Guestbook App. If you are still signed into AppScale, the Guestbook App will use your email address. If you go back to the AppScale Administration Panel and logout, it will sign the guestbook as an anonymous user.

To update an application, simply use the appscale deploy command again. AppScale will automatically detect and update the existing application. You must use the same email address that already owns the application. If you want to change ownership, you can remove and redeploy the application.

If you want to run multiple versions of the same application side-by-side, you will need to change the name of the app in the app.yaml file. This is the main configuration file for the application, and it is located in the root directory of the application.

To remove an application you can use the following command (substitute guestbookgo with the ID AppScale assigned to your app during the deployment process):

appscale remove guestbookgo

You can also remove and deploy your applications from the AppScale Administration Panel.

Troubleshooting

AppScale is a very complicated platform, and things can go wrong. We will cover a few steps you can take to help solve some of the most common errors. It is recommended you read the official AppScale Troubleshooting Page for more details.

If you cannot find a solution to your problem, AppScale has a very active mailing list. Make sure when submitting a topic to the mailing list you include as much detail as possible and a copy of your log files. You will be more likely to receive a quick solution to your problem.

Forcefully Cleaning Up AppScale State

The appscale clean command is used to forcefully bring your VMs to a clean state, removing any configuration problems.

appscale clean

This script will also forcefully kill all AppScale related processes. If you are having problems with an initial deployment, always try this first before contacting the mailing list. This command will usually fix any configuration problems. You can then run appscale up again to re-deploy AppScale.

appscale up

AppScale Log Files

The appscale logs command will gather the logs files from all nodes in an AppScale deployment and copy them to the specified directory.

appscale logs directory/

The log files can be accessed directly in the /var/log/appscale directory.

cd /var/log/appscale

If for some reasons the appscale logs command fails, you will want to access the logs in this manner. However, for a multi-node deployment you will need to do this on every server, which is why it’s recommended you use the AppScale Tools to gather the log files.

The appscale tail command will provide a real-time readout of the AppScale logs in a deployment. This is useful for monitoring application and connection problems in real-time.

appscale tail

Debugging AppScale Deployments

There are three main logs we should be interested in while debugging an AppScale Deployment.

• controller-17443.log — This log is the output of the AppController, the provisioning daemon of AppScale. Since this daemon is responsible for starting all the required services of AppScale, it is the best place start when having problems with an AppScale deployment.

• app___app_id-*.log — Every deployed application will have its own log file. If you are having problems deploying an application, or it is not behaving as expected, this is where you will want to start.

• datastore_server-400*.log — This is the log file for the AppScale datastore.

Conclusion

We have installed and configured AppScale for a single server deployment. We learned how to deploy and remove applications. We also put our deployment to the test by signing the Guestbook App. Signing the Guestbook App proved that a number of APIs are functioning correctly. We can now use this AppScale installation to deploy custom applications based on Google App Engine.

wget -O bootstrap.sh http://bootstrap.appscale.com && bash ./bootstrap.sh