How To Install and Configure AppScale on Ubuntu 12.04
AppScale is an open source computing platform designed to deploy Google App Engine applications on public clouds, private clouds, and on-premise clusters. AppScale is fully compatible with the Google App Engine APIs and has support for Python, Go, PHP, and Java. With AppScale you can migrate existing apps to any cloud compute platform, including DigitalOcean. Below you will find a list of the open source components used to serve a given API.
- Datastore API: Cassandra and ZooKeeper
- Memcache API: memcached
- Task Queue API: RabbitMQ and Celery
- XMPP API: ejabberd
- Channel API: strophe.js and ejabberd
- Blobstore API: Cassandra and ZooKeeper
- Images API: PIL
- Cron API: Vixie Cron
For this tutorial, you will need:
- 4GB+ Droplet with Ubuntu 12.04.5
AppScale requires at least 2 GB of RAM to compile the required components in addition to the 2 GB of RAM that AppScale uses at idle. A minimum of 4 GB of RAM is strongly recommended for standard application deployments. It may be possible to use a 2 GB Droplet with a swap file. However, that is beyond the scope of this tutorial.
At the time of writing, AppScale only has official support for Ubuntu 12.04. If you modify the build script, it may be possible to install on Ubuntu 14.04. However, that is also outside the scope of this tutorial and may not be supported by the community.
The first 2 steps, installing AppScale and the AppScale Tools, must be run as the root user. The remaining steps can be run as a non-root user.
Step 1 — Install AppScale
For the first two sections we will want to run all the commands as the root user. If you are connected to the server as a sudo user, enter the root shell with:
- sudo su
First, update your apt-get package index:
- apt-get update
We are now ready to install AppScale. We will be compiling AppScale from source. Please note compiling source code can be very time consuming. Expect this process to take upwards of 15 minutes or longer to complete.
Ensure you are in the
- cd /root
Install Git so you can use it to download the AppScale source code:
- apt-get install -y git-core
Clone the AppScale source code from GitHub:
- git clone git://github.com/AppScale/appscale.git
Change to the
appscale/debian directory, and run the build script.
Note: This process will take some time. The build script will install any missing dependencies and compile the AppScale source code.
- cd appscale/debian
- bash appscale_build.sh
Step 2 — Install AppScale Tools
The AppScale Tools are used to manage AppScale Clusters and deploy applications. These tools can be installed on a local machine or your server. For simplicity we will be installing the tools on our server. The install process on Mac OS X and Windows is very similar. You need to use Cygwin on Windows. See the GitHub Page for more information.
Change back to the
- cd /root
Clone the AppScale Tools source code from GitHub:
- git clone git://github.com/AppScale/appscale-tools.git
Change to the
appscale-tools/debian directory, and run the build script.
Note: This process will take some time. The build script will install any missing dependencies and compile the AppScale Tools source code.
- cd appscale-tools/debian
- bash appscale_build.sh
After the build script completes, it would be a good idea to reboot.
Step 3 — Configure Your AppScale Deployment
For the remaining portion of this tutorial, you can run the AppScale Tools as any user. This does not need to be a sudo user. However, you will need to know the root user's password when starting AppScale for the first time. AppScale will automatically create authentication certificates, and the root password will no longer be required when working with the AppScale Tools in the future.
After the server has finished rebooting, and you have established your SSH connection, you need to configure your AppScale deployment. The AppScale Tools require a configuration file every time you run the toolset. In this step we will create the configuration file called
AppScalefile, start AppScale, and configure the administrator account.
Ensure you are in your user's home directory:
- cd ~
Create the initial
AppScalefile configuration file:
- appscale init cluster
Now, we will add the server's IP address to the
Open the file with nano:
- nano AppScalefile
At the top of the file, you will see the following section:
# The deployment strategy (roles -> machines) that should be used in this # AppScale deployment. # The following is a sample layout for running everything on one machine: ips_layout : master : your_server_ip appengine : your_server_ip database : your_server_ip zookeeper : your_server_ip
Replace the default IP address with your server's IP address. When you are done editing the file, press CTRL-X, press Y to save, and press ENTER to overwrite the existing file name.
Now we can start AppScale from the same directory as the
AppScalefile just created:
- appscale up
AppScale will ask you to verify the host fingerprint and root password.
The authenticity of host '18.104.22.168 (22.214.171.124)' can't be established. ECDSA key fingerprint is ab:3a:f0:87:c8:4e:8c:ba:59:0e:06:64:1b:f6:fe:e8. Are you sure you want to continue connecting (yes/no)? yes
yes, and press ENTER. You will then see the following:
Enter the root user password, and press ENTER.
After entering the correct root password, you will see the following:
Generated a new SSH key for this deployment at /root/.appscale/appscale69de89364b624a8a9be1b7f45ac23d40 Starting AppScale 2.3.1 over a virtualized cluster. Log in to your head node: ssh -i /root/.appscale/appscale69de89364b624a8a9be1b7f45ac23d40.key email@example.com Head node successfully initialized at 126.96.36.199. It is now starting up cassandra. Copying over deployment credentials Starting AppController at 188.8.131.52 Please wait for the AppController to finish pre-processing tasks. Please wait for AppScale to prepare your machines for use. AppController just started
When starting AppScale, it may seem like it is hung at
AppController just started. This is normal. It can take some time for all the AppScale components to initialize.
Eventually, you will see the following:
UserAppServer is at 184.108.40.206 Enter your desired admin e-mail address:
Create an admin user account. Enter the email address for the user, and provide a password. Remember these details. You will need them to access the AppScale Administration Panel.
After you create the admin user account, you will see:
Creating new user account firstname.lastname@example.org Creating new user account email@example.com Your XMPP username is firstname.lastname@example.org Granting admin privileges to email@example.com AppScale successfully started! View status information about your AppScale deployment at http://220.127.116.11:1080/status
AppScale will provide you with a link to the administration panel. Usually in the following format. Typically the http address will automatically redirect to the secure https address.
Step 4 — The AppScale Administration Panel
Open the AppScale Administration Panel in your browser. The link should have been provided to you after starting AppScale:
You may be prompted to accept the self signed certificate.
From the AppScale Administration Panel, users can create their own accounts by clicking Create Account. However, you will need to change their permissions using the admin account before they can upload and remove their own apps.
Click the Login button in the top right. The Login button may look different on smaller screens, but it will still be green.
Login with the admin email and password you set in the previous step. You will then be presented with the AppScale status page.
The Administration Panel gives you access to server statistics and application statistics. You can also deploy and remove applications. It is fairly straight forward to deploy an application from the Administration Panel. For the purposes of this tutorial we will learn how to deploy an application from the command line. When you are finished exploring the Administration Panel, continue to the next step.
Step 5 — Deploying Your First Application
AppScale provides a collection of sample applications that are ready to deploy. These applications are a good way to test your AppScale cluster. They also familiarize you with the application deployment process.
You should be using the same user account, and your current directory should contain the
AppScaleFile. This file contains all the configurations AppScale requires to manage you deployment.
Make sure we are back in your user's home directory:
- cd ~
From GitHub, clone the sample application source code to create the Guestbook App:
- git clone https://github.com/AppScale/sample-apps.git
You will see the following as the source code downloads:
Cloning into 'sample-apps'... remote: Counting objects: 15742, done. remote: Total 15742 (delta 0), reused 0 (delta 0), pack-reused 15742 Receiving objects: 100% (15742/15742), 318.96 MiB | 23.52 MiB/s, done. Resolving deltas: 100% (4944/4944), done.
The Guestbook App is a great way to test the datastore and authentication APIs.
Deploy the application:
- appscale deploy sample-apps/go/go-guestbook/
You will be asked to assign an email address to your application. Enter the email address, and press Enter.
Enter your desired e-mail address: firstname.lastname@example.org
This can be any email address. If the user does not already exist in the database, you will be prompted to set a password. For the purposes of this tutorial we decided to use the admin account.
Next, you will see the following:
Uploading initial version of app guestbookgo We have reserved guestbookgo for your app Tarring application Copying over application Please wait for your app to start serving. Waiting 1 second(s) to check on application... Waiting 2 second(s) to check on application... Waiting 4 second(s) to check on application... Waiting 8 second(s) to check on application... Waiting 16 second(s) to check on application... Your app can be reached at the following URL: http://18.104.22.168:8080
Open the URL provided in your browser, and you will be served by the Guestbook App. If you are still signed into AppScale, the Guestbook App will use your email address. If you go back to the AppScale Administration Panel and logout, it will sign the guestbook as an anonymous user.
To update an application, simply use the
appscale deploy command again. AppScale will automatically detect and update the existing application. You must use the same email address that already owns the application. If you want to change ownership, you can remove and redeploy the application.
If you want to run multiple versions of the same application side-by-side, you will need to change the name of the app in the
app.yaml file. This is the main configuration file for the application, and it is located in the root directory of the application.
To remove an application you can use the following command (substitute
guestbookgo with the ID AppScale assigned to your app during the deployment process):
- appscale remove guestbookgo
You can also remove and deploy your applications from the AppScale Administration Panel.
AppScale is a very complicated platform, and things can go wrong. We will cover a few steps you can take to help solve some of the most common errors. It is recommended you read the official AppScale Troubleshooting Page for more details.
If you cannot find a solution to your problem, AppScale has a very active mailing list. Make sure when submitting a topic to the mailing list you include as much detail as possible and a copy of your log files. You will be more likely to receive a quick solution to your problem.
Forcefully Cleaning Up AppScale State
appscale clean command is used to forcefully bring your VMs to a clean state, removing any configuration problems.
- appscale clean
This script will also forcefully kill all AppScale related processes. If you are having problems with an initial deployment, always try this first before contacting the mailing list. This command will usually fix any configuration problems. You can then run
appscale up again to re-deploy AppScale.
- appscale up
AppScale Log Files
appscale logs command will gather the logs files from all nodes in an AppScale deployment and copy them to the specified directory.
- appscale logs directory/
The log files can be accessed directly in the
- cd /var/log/appscale
If for some reasons the
appscale logs command fails, you will want to access the logs in this manner. However, for a multi-node deployment you will need to do this on every server, which is why it's recommended you use the AppScale Tools to gather the log files.
appscale tail command will provide a real-time readout of the AppScale logs in a deployment. This is useful for monitoring application and connection problems in real-time.
- appscale tail
Debugging AppScale Deployments
There are three main logs we should be interested in while debugging an AppScale Deployment.
controller-17443.log— This log is the output of the AppController, the provisioning daemon of AppScale. Since this daemon is responsible for starting all the required services of AppScale, it is the best place start when having problems with an AppScale deployment.
app___app_id-*.log— Every deployed application will have its own log file. If you are having problems deploying an application, or it is not behaving as expected, this is where you will want to start.
datastore_server-400*.log— This is the log file for the AppScale datastore.
We have installed and configured AppScale for a single server deployment. We learned how to deploy and remove applications. We also put our deployment to the test by signing the Guestbook App. Signing the Guestbook App proved that a number of APIs are functioning correctly. We can now use this AppScale installation to deploy custom applications based on Google App Engine.