Strona główna Odkrywaj Pomoc
Zarejestruj się Zaloguj się
J1X
/
huginn
Obserwuj 2
Polub 0
Fork 0

Przeglądaj źródła
Add single process docker container

Dominik Sander 8 lat temu
rodzic
73f5564b08
commit
a5417fabf1
8 zmienionych plików z 358 dodań i 0 usunięć
Pokaż statystyki zmian
  1. 16 0
      docker/single-process/Dockerfile
  2. 94 0
      docker/single-process/README.md
  3. 35 0
      docker/single-process/docker-compose.yml
  4. 7 0
      docker/single-process/environment.yml
  5. 72 0
      docker/single-process/postgresql.yml
  6. 57 0
      docker/single-process/scripts/init
  7. 43 0
      docker/single-process/scripts/prepare
  8. 34 0
      docker/single-process/scripts/setup

+ 16 - 0
docker/single-process/Dockerfile
Wyświetl plik 

@@ -0,0 +1,16 @@
1 
+FROM ubuntu:14.04
2 
+MAINTAINER Dominik Sander
3 
+
4 
+ADD scripts/prepare /scripts/prepare
5 
+RUN /scripts/prepare
6 
+
7 
+ADD scripts/setup /scripts/setup
8 
+RUN /scripts/setup
9 
+
10 
+WORKDIR /app
11 
+
12 
+ADD scripts/init /scripts/init
13 
+
14 
+EXPOSE 3000
15 
+
16 
+CMD ["/scripts/init"]

+ 94 - 0
docker/single-process/README.md
Wyświetl plik 

@@ -0,0 +1,94 @@
1 
+Docker image for Huginn using the production environment and separate container for every process
2 
+=================================================
3 
+
4 
+This image runs a linkable [Huginn](https://github.com/cantino/huginn) instance.
5 
+
6 
+It was inspired by the [official docker container for huginn](https://registry.hub.docker.com/u/cantino/huginn)
7 
+
8 
+The scripts/init script generates a .env file containing the variables as passed as per normal Huginn documentation.
9 
+The same environment variables that would be used for Heroku PaaS deployment are used by this script.
10 
+
11 
+The scripts/init script is aware of mysql and postgres linked containers through the environment variables:
12 
+
13 
+    MYSQL_PORT_3306_TCP_ADDR
14 
+    MYSQL_PORT_3306_TCP_PORT
15 
+
16 
+and
17 
+
18 
+    POSTGRESQL_PORT_5432_TCP_ADDR
19 
+    POSTGRESQL_PORT_5432_TCP_PORT
20 
+
21 
+Its recommended to use an image that allows you to create a database via environmental variables at docker run, like `postgresql` or `mysql`, so the db is populated when this script runs.
22 
+
23 
+Additionally, the database variables may be overridden from the above as per the standard Huginn documentation:
24 
+
25 
+    DATABASE_ADAPTER #(must be either 'postgresql' or 'mysql2')
26 
+    DATABASE_HOST
27 
+    DATABASE_PORT
28 
+
29 
+This script will run database migrations (rake db:migrate) which should be idempotent.
30 
+
31 
+It will also seed the database (rake db:seed) unless this is defined:
32 
+
33 
+    DO_NOT_SEED
34 
+
35 
+This same seeding initially defines the "admin" user with a default password of "password" as per the standard Huginn documentation.
36 
+
37 
+If you do not wish to have the default 6 agents, you will want to set the above environment variable after your initially deploy, otherwise they will be added automatically the next time a container pointing at the database is spun up.
38 
+
39 
+The CMD launches Huginn via the scripts/init script. This may become the ENTRYPOINT later.  It does take under a minute for Huginn to come up.  Use environmental variables that match your DB's creds to ensure it works.
40 
+
41 
+## Usage
42 
+
43 
+Simple startup using docker compose (you need to daemonize with `-d` to persist the data):
44 
+
45 
+    cd docker/single-process
46 
+    docker-compose up
47 
+
48 
+or if you like to use PostgreSQL:
49 
+
50 
+    docker-compose -f postgresql.yml up
51 
+
52 
+Manual startup and linking to a MySQL container:
53 
+
54 
+    docker run --name huginn_mysql \
55 
+        -e MYSQL_DATABASE=huginn \
56 
+        -e MYSQL_USER=huginn \
57 
+        -e MYSQL_PASSWORD=somethingsecret \
58 
+        -e MYSQL_ROOT_PASSWORD=somethingevenmoresecret \
59 
+        mysql
60 
+
61 
+    docker run --name huginn_web \
62 
+        --link huginn_mysql:mysql \
63 
+        -p 3000:3000 \
64 
+        -e DATABASE_NAME=huginn \
65 
+        -e DATABASE_USERNAME=huginn \
66 
+        -e DATABASE_PASSWORD=somethingsecret \
67 
+        dsander/huginn-single-process
68 
+
69 
+    docker run --name huginn_threaded \
70 
+        --link huginn_mysql:mysql \
71 
+        -e DATABASE_NAME=huginn \
72 
+        -e DATABASE_USERNAME=huginn \
73 
+        -e DATABASE_PASSWORD=somethingsecret \
74 
+        dsander/huginn-single-process /scripts/init bin/threaded.rb
75 
+
76 
+## Environment Variables
77 
+
78 
+Other Huginn 12factored environment variables of note, as generated and put into the .env file as per Huginn documentation. All variables of the [.env.example](https://github.com/cantino/huginn/blob/master/.env.example) can be used to override the defaults which a read from the current `.env.example`.
79 
+
80 
+For variables in the .env.example that are commented out, the default is to not include that variable in the generated .env file.
81 
+
82 
+## Building on your own
83 
+
84 
+You don't need to do this on your own, but if you really want:
85 
+
86 
+    docker build --rm=true --tag={yourname}/huginn .
87 
+
88 
+## Source
89 
+
90 
+The source is [available on GitHub](https://github.com/cantino/huginn/docker/single-process/).
91 
+
92 
+Please feel free to submit pull requests and/or fork at your leisure.
93 
+
94 
+

+ 35 - 0
docker/single-process/docker-compose.yml
Wyświetl plik 

@@ -0,0 +1,35 @@
1 
+mysqldata:
2 
+  image: mysql
3 
+  command: /bin/true
4 
+
5 
+mysql:
6 
+  image: mysql
7 
+  volumes_from:
8 
+    - mysqldata
9 
+  environment:
10 
+    MYSQL_ROOT_PASSWORD: myrootpassword
11 
+    MYSQL_DATABASE: huginn
12 
+    MYSQL_USER: huginn
13 
+    MYSQL_PASSWORD: myhuginnpassword
14 
+
15 
+huginn_web:
16 
+  build: .
17 
+  restart: always
18 
+  extends:
19 
+    file: environment.yml
20 
+    service: huginn_base
21 
+  ports:
22 
+    - 3000:3000
23 
+  links:
24 
+    - mysql
25 
+
26 
+huginn_threaded:
27 
+  build: .
28 
+  restart: always
29 
+  extends:
30 
+    file: environment.yml
31 
+    service: huginn_base
32 
+  links:
33 
+    - mysql
34 
+  command: /scripts/init bin/threaded.rb
35 
+

+ 7 - 0
docker/single-process/environment.yml
Wyświetl plik 

@@ -0,0 +1,7 @@
1 
+huginn_base:
2 
+  environment:
3 
+    DATABASE_ADAPTER: mysql2
4 
+    DATABASE_NAME: huginn
5 
+    DATABASE_USERNAME: huginn
6 
+    DATABASE_PASSWORD: myhuginnpassword
7 
+    APP_SECRET_TOKEN: 3bd139f9186b31a85336bb89cd1a1337078921134b2f48e022fd09c234d764d3e19b018b2ab789c6e0e04a1ac9e3365116368049660234c2038dc9990513d49c

+ 72 - 0
docker/single-process/postgresql.yml
Wyświetl plik 

@@ -0,0 +1,72 @@
1 
+postgresdata:
2 
+  image: postgres
3 
+  command: /bin/true
4 
+
5 
+postgres:
6 
+  image: postgres
7 
+  volumes_from:
8 
+    - postgresdata
9 
+  environment:
10 
+    POSTGRES_PASSWORD: myhuginnpassword
11 
+    POSTGRES_USER: huginn
12 
+
13 
+huginn_web:
14 
+  build: .
15 
+  restart: always
16 
+  extends:
17 
+    file: environment.yml
18 
+    service: huginn_base
19 
+  environment:
20 
+    DATABASE_ADAPTER: postgresql
21 
+  ports:
22 
+    - 3000:3000
23 
+  links:
24 
+    - postgres
25 
+
26 
+huginn_threaded:
27 
+  build: .
28 
+  restart: always
29 
+  extends:
30 
+    file: environment.yml
31 
+    service: huginn_base
32 
+  environment:
33 
+    DATABASE_ADAPTER: postgresql
34 
+  links:
35 
+    - postgres
36 
+  command: /scripts/init bin/threaded.rb
37 
+
38 
+# huginn_schedule:
39 
+#   build: .
40 
+#   extends:
41 
+#     file: environment.yml
42 
+#     service: huginn_base
43 
+#   environment:
44 
+#     DATABASE_ADAPTER: postgresql
45 
+#   links:
46 
+#     - postgres
47 
+#   command: /scripts/init bin/schedule.rb
48 
+
49 
+
50 
+# huginn_twitter_stream:
51 
+#   build: .
52 
+#   extends:
53 
+#     file: environment.yml
54 
+#     service: huginn_base
55 
+#   environment:
56 
+#     DATABASE_ADAPTER: postgresql
57 
+#   links:
58 
+#     - postgres
59 
+#   command: /scripts/init bin/twitter_stream.rb
60 
+
61 
+
62 
+# huginn_dj1:
63 
+#   build: .
64 
+#   extends:
65 
+#     file: environment.yml
66 
+#     service: huginn_base
67 
+#   environment:
68 
+#     DATABASE_ADAPTER: postgresql
69 
+#   links:
70 
+#     - postgres
71 
+#   command: /scripts/init script/delayed_job run
72 
+

+ 57 - 0
docker/single-process/scripts/init
Wyświetl plik 

@@ -0,0 +1,57 @@
1 
+#!/bin/bash
2 
+set -e
3 
+
4 
+cd /app
5 
+
6 
+# Configure database based on linked container
7 
+if [ -n "${MYSQL_PORT_3306_TCP_ADDR}" ]; then
8 
+  DATABASE_ADAPTER=${DATABASE_ADAPTER:-mysql2}
9 
+  DATABASE_HOST=${DATABASE_HOST:-${MYSQL_PORT_3306_TCP_ADDR}}
10 
+  DATABASE_PORT=${DATABASE_PORT:-${MYSQL_PORT_3306_TCP_PORT}}
11 
+  DATABASE_ENCODING=${DATABASE_ENCODING:-utf8mb4}
12 
+elif [ -n "${POSTGRES_PORT_5432_TCP_ADDR}" ]; then
13 
+  DATABASE_ADAPTER=${DATABASE_ADAPTER:-postgresql}
14 
+  DATABASE_HOST=${DATABASE_HOST:-${POSTGRES_PORT_5432_TCP_ADDR}}
15 
+  DATABASE_PORT=${DATABASE_PORT:-${POSTGRES_PORT_5432_TCP_PORT}}
16 
+  DATABASE_ENCODING=utf8
17 
+fi
18 
+
19 
+USE_GRAPHVIZ_DOT=${USE_GRAPHVIZ_DOT:-${USE_GRAPHVIZ_DOT:-dot}}
20 
+
21 
+# Default to the environment variable values set in .env.example
22 
+IFS="="
23 
+grep = /app/.env.example | sed -e 's/^#\([^ ]\)/\1/' | grep -v -e '^#' | \
24 
+  while read var value ; do
25 
+    eval "echo \"$var=\${$var:-\${HUGINN_$var-\$value}}\""
26 
+  done | grep -v -e ^= > /app/.env
27 
+
28 
+eval "echo PORT=${PORT:-${PORT:-3000}}" >> .env
29 
+eval "echo RAILS_ENV=${RAILS_ENV:-${RAILS_ENV:-production}}" >> .env
30 
+eval "echo ON_HEROKU=true" >> .env
31 
+eval "echo RAILS_SERVE_STATIC_FILES=true" >> .env
32 
+
33 
+chmod ugo+r /app/.env
34 
+source /app/.env
35 
+
36 
+# use default port number if it is still not set
37 
+case "${DATABASE_ADAPTER}" in
38 
+  mysql2) DATABASE_PORT=${DATABASE_PORT:-3306} ;;
39 
+  postgresql) DATABASE_PORT=${DATABASE_PORT:-5432} ;;
40 
+  *) echo "Unsupported database adapter. Available adapters are mysql2, and postgresql." && exit 1 ;;
41 
+esac
42 
+
43 
+sudo -u huginn -H bundle install --without test development --path vendor/bundle
44 
+
45 
+if [ -z $1 ]; then
46 
+  sudo -u huginn -H bundle exec rake db:create db:migrate RAILS_ENV=${RAILS_ENV}
47 
+fi
48 
+
49 
+if [[ -z "${DO_NOT_SEED}" && -z $1 ]]; then
50 
+  sudo -u huginn -H bundle exec rake db:seed RAILS_ENV=${RAILS_ENV}
51 
+fi
52 
+
53 
+if [ -z $1 ]; then
54 
+  exec sudo -u huginn -H bundle exec unicorn -c config/unicorn.rb
55 
+else
56 
+  exec sudo -u huginn -H bundle exec rails runner "$@" RAILS_ENV=${RAILS_ENV}
57 
+fi

+ 43 - 0
docker/single-process/scripts/prepare
Wyświetl plik 

@@ -0,0 +1,43 @@
1 
+#!/bin/bash
2 
+set -e
3 
+
4 
+cat > /etc/dpkg/dpkg.cfg.d/01_nodoc <<EOF
5 
+# Delete locales
6 
+path-exclude=/usr/share/locale/*
7 
+
8 
+# Delete man pages
9 
+path-exclude=/usr/share/man/*
10 
+
11 
+# Delete docs
12 
+path-exclude=/usr/share/doc/*
13 
+path-include=/usr/share/doc/*/copyright
14 
+EOF
15 
+
16 
+cat > /etc/default/locale <<EOF
17 
+LC_ALL=en_US.UTF-8
18 
+LANG=en_US.UTF-8
19 
+EOF
20 
+
21 
+export LC_ALL=C
22 
+export DEBIAN_FRONTEND=noninteractive
23 
+minimal_apt_get_install='apt-get install -y --no-install-recommends'
24 
+
25 
+apt-get update
26 
+apt-get dist-upgrade -y --no-install-recommends
27 
+$minimal_apt_get_install software-properties-common
28 
+add-apt-repository -y ppa:brightbox/ruby-ng
29 
+apt-get update
30 
+$minimal_apt_get_install build-essential checkinstall git-core \
31 
+  zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev \
32 
+  libncurses5-dev libffi-dev libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev \
33 
+  graphviz libgraphviz-dev \
34 
+  libmysqlclient-dev libpq-dev libsqlite3-dev \
35 
+  ruby2.2 ruby2.2-dev
36 
+gem install --no-ri --no-rdoc bundler
37 
+
38 
+apt-get purge -y python3* rsyslog rsync manpages
39 
+rm -rf /var/lib/apt/lists/*
40 
+rm -rf /usr/share/doc/
41 
+rm -rf /usr/share/man/
42 
+rm -rf /usr/share/locale/
43 
+rm -rf /var/log/*

+ 34 - 0
docker/single-process/scripts/setup
Wyświetl plik 

@@ -0,0 +1,34 @@
1 
+#!/bin/bash
2 
+set -e
3 
+
4 
+# add a huginn group and user
5 
+adduser --group huginn
6 
+adduser --disabled-login --ingroup huginn --gecos 'Huginn' --no-create-home --home /app huginn
7 
+passwd -d huginn
8 
+
9 
+# Shallow clone the huginn project repo
10 
+git clone --depth 1 https://github.com/cantino/huginn /app
11 
+
12 
+# Change the ownership to huginn
13 
+chown -R huginn:huginn /app
14 
+
15 
+cd app
16 
+
17 
+# create required tmp and log directories
18 
+sudo -u huginn -H mkdir -p tmp/pids tmp/cache tmp/sockets log
19 
+chmod -R u+rwX log tmp
20 
+
21 
+# HACK: We need a database connection to precompile the assets, use sqlite for that
22 
+echo "gem 'sqlite3', '~> 1.3.11'" >> Gemfile
23 
+sudo -u huginn -H RAILS_ENV=production APP_SECRET_TOKEN=secret DATABASE_ADAPTER=sqlite3 ON_HEROKU=true bundle install --without test development --path vendor/bundle -j 4
24 
+sudo -u huginn -H RAILS_ENV=production APP_SECRET_TOKEN=secret DATABASE_ADAPTER=sqlite3 ON_HEROKU=true bundle exec rake assets:clean assets:precompile
25 
+git checkout Gemfile
26 
+
27 
+# Bundle again to get rid of the sqlite3 gem
28 
+sudo -u huginn -H ON_HEROKU=true DATABASE_ADAPTER=noop bundle install --without test development --path vendor/bundle
29 
+
30 
+# Configure the unicorn server
31 
+mv config/unicorn.rb.example config/unicorn.rb
32 
+sed -ri 's/^listen .*$/listen ENV["PORT"]/' config/unicorn.rb
33 
+sed -ri 's/^stderr_path.*$//' config/unicorn.rb
34 
+sed -ri 's/^stdout_path.*$//' config/unicorn.rb