Recursive Thoughts

How to deploy BotBot on Digital Ocean

Claudia Doppioslash -

I recently started ##narrative-ai on freenode, and I find myself in need of a IRC logger. Preferably a not-ugly IRC logger.

The most mature seems to be BotBot, they're supposed to host it for free.
But there's no guarantee they'll accept my channel, and I have already waited for more than a week, so here I am deploying it on a Digital Ocean server. There are quite a few moving parts to it.

You'll need:

  • Ubuntu
  • Redis
  • PostgreSQL
  • Python
  • Go
  • gunicorn
  • nginx

I think it best to get a droplet with redis already installed, to save some time and effort.

Droplet setup

Once you have that there is the usual droplet ssh dance:

  • create new user
$ adduser <bla>
  • make it sudoable
$ adduser <bla> sudo
  • install mosh (mosh is a better ssh, it's quite great but it's up to you if you want to do this)
$ sudo apt-get install mosh
  • logout and get back in with your new user
$ logout
$ mosh <bla>@droplet-ip

so that's the basic droplet setup.

BotBot

Now for botbot:

Get dependencies

You should install the deps

$ sudo apt-get install postgresql-contrib-9.3 postgresql-server-dev-9.3 python-dev python-pip golang-go git

you need pip to install virtualenv

$ sudo pip install virtualenv

if you didn't use the redis droplet you'll need to get redis as well

$ sudo apt-get install redis-server

Setup Postgres

Now, postgres needs some setup:

login with the postgres user

$ sudo -i -u postgres

change user e pass on the postgresql

createuser -P -s -e botbot

This particular version of the createuser incantation will prompt for password, sparing you from having to find out how to change it later.

That should have sorted postgres.

Install BotBot

Now you can start to install botbot itself.
From the botbot install guide:

$ virtualenv botbot && source botbot/bin/activate

$ pip install -e git+https://github.com/BotBotMe/botbot-web.git#egg=botbot

$ cd $VIRTUAL_ENV/src/botbot

So now you have a python environment for your botbot install.
Nest step is to get the dependencies. That entails compiling and will take a while... go make a brew maybe?

$ make dependencies

Now you have to get the specifics of your environment in the .env file.

$ vim .env

# Required
SECRET_KEY=supersecretkeyhere
WEB_PORT=8000
EMAIL_BACKEND='django.core.mail.backends.console.EmailBackend'
GOPATH=$VIRTUAL_ENV
WEB_SECRET_KEY=somerandomstring
STORAGE_URL=postgres://<user>:<pass>@localhost:5432/botbot
REDIS_PLUGIN_STORAGE_URL=redis://localhost:6379/0
REDIS_PLUGIN_QUEUE_URL=redis://localhost:6379/1
PUSH_STREAM_URL=http://localhost:8080/pub/?id={id}

# Set encoding if the system hasn't done it properly
LANG=en_US.UTF-8
PYTHONIOENCODING=utf8

The next command makes the values in .env available, you should run it any time you make changes to .env:

$ export $(cat .env | grep -v ^# | xargs)

Now the db setup:

createdb botbot
echo "create extension hstore" | psql botbot
manage.py migrate

It will likely complain that you don't have geoip-db, if so:

$ make geoip-db

And now the last command:

# You'll need a staff account for creating a bot and registering channels
manage.py createsuperuser

botbot-web

Try out your install with:

honcho start

you may need first to change the Procfile changing $WEB_PORT to something like 0.0.0.0:8000 so you can connect from outside the machine.

If everything went well you should see django admin page connecting to http://<machine ip>:port/admin.

Follow the getting started instructions to set up your bot.

If you're happy with honcho start that's it.

But if you want to have upstart services and nginx go on reading.

Production setup

botbot-plugins and botbot-bot

In the instructions you have two example upstart script, one for the plugins and one for the irc bot. You need to customise them with paths, usernames and passwords:

$ vim botbot-bot.conf

# BotBot-bot
# logs to /var/log/upstart/botbot.log

description "BotBot"
start on startup
stop on shutdown

respawn
env LANG=en_US.UTF-8
env STORAGE_URL=postgres://<user>:<pass>:5432/botbot
env REDIS_PLUGIN_QUEUE_URL=redis://localhost:6379/0

exec <path/to>/botbot/bin/botbot-bot
setuid www-data

$ vim botbot-plugins.conf
# BotBot Plugins
# logs to /var/log/upstart/botbot_plugins.log

description "BotBot Plugins"
start on startup
stop on shutdown

respawn
env LANG=en_US.UTF-8
exec <path/to>/botbot/bin/manage.py run_plugins
setuid www-data

You might to want to check that they are valid by using:

$ init-checkconf <file>.conf

Then you have to move or copy them to /etc/init/ where upstart can find them:

$ sudo cp botbot-*.conf /etc/init/

You should be able to start them using:

$ sudo service botbot-plugins start && sudo service botbot-bot start

obtaining something like:

botbot-plugins start/running, process 10612
botbot-bot start/running, process 10621

Last but not least, now we should setup wsgi serving of the web interface.

The instructions say:

botbot-web: should be served as a wsgi application, from the wsgi.py file located at src/botbot/botbot/wsgi.py from uwsgi, gunicorn, mod_wsgi, or any other wsgi server.

Sadly enough we need either Apache with mod_wsgi or nginx with gunicorn or uwsgi.

I opted for nginx+gunicorn exlusively based on the fact that it looks like less hassle than ngix+wsgi or Apache2+mod_wsgi xD

pip install gunicorn

sudo apt-get install nginx

There is an example nginx.conf file included in botbot-web.

gunicorn -b 0.0.0.0:8080 wsgi

To test that it works

Refs