Начало Разгледай Помощ
Регистриране Влизане
J1X
/
huginn
Следи 2
Харесвам 0
Разклонение 0

Преглед на кода
Merge pull request #1204 from dsander/docker

Use production env in Docker image, add single process Docker image

Dominik Sander преди 8 години
родител
91d69b2a37 95955b7b5d
ревизия
89a824cffa
променени са 19 файла, в които са добавени 559 реда и са изтрити 245 реда
Покажи статистика за разликите
  1. 0 36
      docker/Dockerfile
  2. 7 147
      docker/README.md
  3. 16 0
      docker/multi-process/Dockerfile
  4. 0 0
      docker/multi-process/Dockerfile.rbenv
  5. 0 0
      docker/multi-process/Makefile
  6. 103 0
      docker/multi-process/README.md
  7. 0 0
      docker/multi-process/docker-compose.yml
  8. 32 24
      docker/scripts/init
  9. 9 0
      docker/multi-process/scripts/standalone-packages
  10. 0 38
      docker/scripts/setup
  11. 16 0
      docker/single-process/Dockerfile
  12. 94 0
      docker/single-process/README.md
  13. 35 0
      docker/single-process/develop.yml
  14. 35 0
      docker/single-process/docker-compose.yml
  15. 7 0
      docker/single-process/environment.yml
  16. 72 0
      docker/single-process/postgresql.yml
  17. 57 0
      docker/single-process/scripts/init
  18. 40 0
      docker/single-process/scripts/prepare
  19. 36 0
      docker/single-process/scripts/setup

+ 0 - 36
docker/Dockerfile
Преглед на файла 

@@ -1,36 +0,0 @@
1 
-FROM ubuntu:14.04
2 
-MAINTAINER Andrew Cantino
3 
-
4 
-ENV DEBIAN_FRONTEND noninteractive
5 
-RUN apt-get update && \
6 
-    apt-get install -y software-properties-common && \
7 
-    add-apt-repository -y ppa:git-core/ppa && \
8 
-    add-apt-repository -y ppa:brightbox/ruby-ng && \
9 
-    apt-get update && \
10 
-    apt-get install -y build-essential checkinstall postgresql-client \
11 
-      git-core mysql-server redis-server python2.7 python-docutils \
12 
-      libmysqlclient-dev libpq-dev zlib1g-dev libyaml-dev libssl-dev \
13 
-      libgdbm-dev libreadline-dev libncurses5-dev libffi-dev \
14 
-      libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev \
15 
-      graphviz libgraphviz-dev \
16 
-      ruby2.1 ruby2.1-dev supervisor python-pip && \
17 
-    gem install --no-ri --no-rdoc bundler && \
18 
-    rm -rf /var/lib/apt/lists/*
19 
-
20 
-RUN pip install supervisor-stdout
21 
-
22 
-ADD scripts/setup /scripts/setup
23 
-RUN chmod 755 /scripts/setup
24 
-RUN /scripts/setup
25 
-
26 
-WORKDIR /app
27 
-
28 
-ADD scripts/init /scripts/init
29 
-RUN chmod 755 /scripts/init
30 
-
31 
-VOLUME /var/lib/mysql
32 
-
33 
-EXPOSE 3000
34 
-
35 
-CMD ["/scripts/init"]
36 
-

+ 7 - 147
docker/README.md
Преглед на файла 

@@ -1,152 +1,12 @@
1 
-Huginn for docker with multiple container linkage
2 
-=================================================
1 
+Huginn Docker images
2 
+====================
3 3
4 
-This image runs a linkable [Huginn](https://github.com/cantino/huginn) instance.
4 
+Huginn is packaged in two docker images.
5 5
6 
-There is an automated build repository on docker hub for [cantino/huginn](https://registry.hub.docker.com/builds/github/cantino/huginn/).
6 
+#### `cantino/huginn` multiple process image
7 7
8 
-This was patterned after [sameersbn/gitlab](https://registry.hub.docker.com/u/sameersbn/gitlab) by [ianblenke/huginn](http://github.com/ianblenke/huginn), and imported here for official generation of a docker hub auto-build image.
9 
-
10 
-The scripts/init script generates a .env file containing the variables as passed as per normal Huginn documentation.
11 
-The same environment variables that would be used for Heroku PaaS deployment are used by this script.
12 
-
13 
-The scripts/init script is aware of mysql and postgres linked containers through the environment variables:
14 
-
15 
-    MYSQL_PORT_3306_TCP_ADDR
16 
-    MYSQL_PORT_3306_TCP_PORT
17 
-
18 
-and
19 
-
20 
-    POSTGRESQL_PORT_5432_TCP_ADDR
21 
-    POSTGRESQL_PORT_5432_TCP_PORT
22 
-
23 
-Its recommended to use an image that allows you to create a database via environmental variables at docker run, like `paintedfox / postgresql` or `centurylink / mysql`, so the db is populated when this script runs.
24 
-
25 
-If you do not link a database container, a built-in mysql database will be started.
26 
-There is an exported docker volume of /var/lib/mysql to allow persistence of that mysql database.
27 
-
28 
-Additionally, the database variables may be overridden from the above as per the standard Huginn documentation:
29 
-
30 
-    HUGINN_DATABASE_ADAPTER #(must be either 'postgres' or 'mysql2')
31 
-    HUGINN_DATABASE_HOST
32 
-    HUGINN_DATABASE_PORT
33 
-
34 
-This script will run database migrations (rake db:migrate) which should be idempotent.
35 
-
36 
-It will also seed the database (rake db:seed) unless this is defined:
37 
-
38 
-    DO_NOT_SEED
39 
-
40 
-This same seeding initially defines the "admin" user with a default password of "password" as per the standard Huginn documentation.
41 
-
42 
-If you do not wish to have the default 6 agents, you will want to set the above environment variable after your initially deploy, otherwise they will be added automatically the next time a container pointing at the database is spun up.
43 
-
44 
-The CMD launches Huginn via the scripts/init script. This may become the ENTRYPOINT later.  It does take under a minute for Huginn to come up.  Use environmental variables that match your DB's creds to ensure it works.
45 
-
46 
-## Usage
47 
-
48 
-Simple stand-alone usage:
49 
-
50 
-    docker run -it -p 3000:3000 cantino/huginn
51 
-
52 
-To link to another mysql container, for example:
53 
-
54 
-    docker run --rm --name huginn_mysql \
55 
-        -e MYSQL_DATABASE=huginn \
56 
-        -e MYSQL_USER=huginn \
57 
-        -e MYSQL_PASSWORD=somethingsecret \
58 
-        -e MYSQL_ROOT_PASSWORD=somethingevenmoresecret \
59 
-        mysql
60 
-    docker run --rm --name huginn \
61 
-        --link huginn_mysql:mysql \
62 
-        -p 3000:3000 \
63 
-        -e HUGINN_DATABASE_NAME=huginn \
64 
-        -e HUGINN_DATABASE_USERNAME=huginn \
65 
-        -e HUGINN_DATABASE_PASSWORD=somethingsecret \
66 
-        cantino/huginn
67 
-
68 
-To link to another container named 'postgres':
69 
-
70 
-    docker run --name huginn_postgres \
71 
-        -e POSTGRES_PASSWORD=mysecretpassword \
72 
-        -e POSTGRES_USER=huginn -d postgres
73 
-    docker run -it --link huginn_postgres:postgres --rm postgres \
74 
-        sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U huginn -c "create database huginn_development;"'
75 
-    docker run --rm --name huginn \
76 
-        --link huginn_postgres:postgres \
77 
-        -p 3000:3000 \
78 
-        -e HUGINN_DATABASE_USERNAME=huginn \
79 
-        -e HUGINN_DATABASE_PASSWORD=mysecretpassword \
80 
-        -e HUGINN_DATABASE_ADAPTER=postgresql \
81 
-        cantino/huginn
82 
-
83 
-The `docker/` folder also has a `docker-compose.yml` that allows for a sample database formation with a data volume container:
84 
-
85 
-    cd docker ; docker-compose up
86 
-
87 
-## Environment Variables
88 
-
89 
-Other Huginn 12factored environment variables of note, as generated and put into the .env file as per Huginn documentation,
90 
-with an additional `HUGINN_` prefix to the variable.
91 
-
92 
-These are:
93 
-
94 
-    HUGINN_APP_SECRET_TOKEN
95 
-    HUGINN_DOMAIN
96 
-    HUGINN_ASSET_HOST
97 
-    HUGINN_DATABASE_ADAPTER
98 
-    HUGINN_DATABASE_ENCODING
99 
-    HUGINN_DATABASE_RECONNECT
100 
-    HUGINN_DATABASE_NAME
101 
-    HUGINN_DATABASE_POOL
102 
-    HUGINN_DATABASE_USERNAME
103 
-    HUGINN_DATABASE_PASSWORD
104 
-    HUGINN_DATABASE_HOST
105 
-    HUGINN_DATABASE_PORT
106 
-    HUGINN_DATABASE_SOCKET
107 
-    HUGINN_RAILS_ENV
108 
-    HUGINN_FORCE_SSL
109 
-    HUGINN_INVITATION_CODE
110 
-    HUGINN_SMTP_DOMAIN
111 
-    HUGINN_SMTP_USER_NAME
112 
-    HUGINN_SMTP_PASSWORD
113 
-    HUGINN_SMTP_SERVER
114 
-    HUGINN_SMTP_PORT
115 
-    HUGINN_SMTP_AUTHENTICATION
116 
-    HUGINN_SMTP_ENABLE_STARTTLS_AUTO
117 
-    HUGINN_EMAIL_FROM_ADDRESS
118 
-    HUGINN_AGENT_LOG_LENGTH
119 
-    HUGINN_TWITTER_OAUTH_KEY
120 
-    HUGINN_TWITTER_OAUTH_SECRET
121 
-    HUGINN_THIRTY_SEVEN_SIGNALS_OAUTH_KEY
122 
-    HUGINN_THIRTY_SEVEN_SIGNALS_OAUTH_SECRET
123 
-    HUGINN_AWS_ACCESS_KEY_ID
124 
-    HUGINN_AWS_ACCESS_KEY
125 
-    HUGINN_AWS_SANDBOX
126 
-    HUGINN_FARADAY_HTTP_BACKEND
127 
-    HUGINN_DEFAULT_HTTP_USER_AGENT
128 
-    HUGINN_ALLOW_JSONPATH_EVAL
129 
-    HUGINN_ENABLE_INSECURE_AGENTS
130 
-    HUGGIN_ENABLE_SECOND_PRECISION_SCHEDULE
131 
-    HUGINN_USE_GRAPHVIZ_DOT
132 
-    HUGINN_TIMEZONE
133 
-    HUGGIN_FAILED_JOBS_TO_KEEP
134 
-
135 
-
136 
-The above environment variables will override the defaults. The defaults are read from the [.env.example](https://github.com/cantino/huginn/blob/master/.env.example) file.
137 
-
138 
-For variables in the .env.example that are commented out, the default is to not include that variable in the generated .env file.
139 
-
140 
-## Building on your own
141 
-
142 
-You don't need to do this on your own, because there is an [automated build](https://registry.hub.docker.com/u/cantino/huginn/) for this repository, but if you really want:
143 
-
144 
-    docker build --rm=true --tag={yourname}/huginn .
145 
-
146 
-## Source
147 
-
148 
-The source is [available on GitHub](https://github.com/cantino/huginn/).
149 
-
150 
-Please feel free to submit pull requests and/or fork at your leisure.
8 
+This image runs all processes needed by Huginn in one container, when not database is linked it will also start MySQL internally. It is great to try huginn without having to set up anything, however maintenance and backups can be difficult.
151 9
10 
+#### `cantino/huginn-single-process` multiple container image
152 11
12 
+This image runs just one process per container and thus needs at least two container to be started, one for the Huginn application server and one for the threaded background worker. It is also possible to every background worker in a separate container to improve the performance. See [the PostgreSQL docker-compose configuration](single-process/postgres.yml) for an example.

+ 16 - 0
docker/multi-process/Dockerfile
Преглед на файла 

@@ -0,0 +1,16 @@
1 
+FROM cantino/huginn-single-process:latest
2 
+MAINTAINER Andrew Cantino
3 
+
4 
+WORKDIR /app
5 
+
6 
+ADD scripts/standalone-packages /scripts/standalone-packages
7 
+RUN /scripts/standalone-packages
8 
+
9 
+ADD scripts/init /scripts/init
10 
+
11 
+VOLUME /var/lib/mysql
12 
+
13 
+EXPOSE 3000
14 
+
15 
+CMD ["/scripts/init"]
16 
+

docker/Dockerfile.rbenv → docker/multi-process/Dockerfile.rbenv
Преглед на файла


docker/Makefile → docker/multi-process/Makefile
Преглед на файла


+ 103 - 0
docker/multi-process/README.md
Преглед на файла 

@@ -0,0 +1,103 @@
1 
+Huginn for docker with multiple container linkage
2 
+=================================================
3 
+
4 
+This image runs a linkable [Huginn](https://github.com/cantino/huginn) instance.
5 
+
6 
+There is an automated build repository on docker hub for [cantino/huginn](https://registry.hub.docker.com/builds/github/cantino/huginn/).
7 
+
8 
+This was patterned after [sameersbn/gitlab](https://registry.hub.docker.com/u/sameersbn/gitlab) by [ianblenke/huginn](http://github.com/ianblenke/huginn), and imported here for official generation of a docker hub auto-build image.
9 
+
10 
+The scripts/init script generates a .env file containing the variables as passed as per normal Huginn documentation.
11 
+The same environment variables that would be used for Heroku PaaS deployment are used by this script.
12 
+
13 
+The scripts/init script is aware of mysql and postgres linked containers through the environment variables:
14 
+
15 
+    MYSQL_PORT_3306_TCP_ADDR
16 
+    MYSQL_PORT_3306_TCP_PORT
17 
+
18 
+and
19 
+
20 
+    POSTGRESQL_PORT_5432_TCP_ADDR
21 
+    POSTGRESQL_PORT_5432_TCP_PORT
22 
+
23 
+Its recommended to use an image that allows you to create a database via environmental variables at docker run, like `paintedfox / postgresql` or `centurylink / mysql`, so the db is populated when this script runs.
24 
+
25 
+If you do not link a database container, a built-in mysql database will be started.
26 
+There is an exported docker volume of /var/lib/mysql to allow persistence of that mysql database.
27 
+
28 
+Additionally, the database variables may be overridden from the above as per the standard Huginn documentation:
29 
+
30 
+    DATABASE_ADAPTER #(must be either 'postgres' or 'mysql2')
31 
+    DATABASE_HOST
32 
+    DATABASE_PORT
33 
+
34 
+This script will run database migrations (rake db:migrate) which should be idempotent.
35 
+
36 
+It will also seed the database (rake db:seed) unless this is defined:
37 
+
38 
+    DO_NOT_SEED
39 
+
40 
+This same seeding initially defines the "admin" user with a default password of "password" as per the standard Huginn documentation.
41 
+
42 
+If you do not wish to have the default 6 agents, you will want to set the above environment variable after your initially deploy, otherwise they will be added automatically the next time a container pointing at the database is spun up.
43 
+
44 
+The CMD launches Huginn via the scripts/init script. This may become the ENTRYPOINT later.  It does take under a minute for Huginn to come up.  Use environmental variables that match your DB's creds to ensure it works.
45 
+
46 
+## Usage
47 
+
48 
+Simple stand-alone usage:
49 
+
50 
+    docker run -it -p 3000:3000 cantino/huginn
51 
+
52 
+To link to another mysql container, for example:
53 
+
54 
+    docker run --rm --name huginn_mysql \
55 
+        -e MYSQL_DATABASE=huginn \
56 
+        -e MYSQL_USER=huginn \
57 
+        -e MYSQL_PASSWORD=somethingsecret \
58 
+        -e MYSQL_ROOT_PASSWORD=somethingevenmoresecret \
59 
+        mysql
60 
+    docker run --rm --name huginn \
61 
+        --link huginn_mysql:mysql \
62 
+        -p 3000:3000 \
63 
+        -e HUGINN_DATABASE_NAME=huginn \
64 
+        -e HUGINN_DATABASE_USERNAME=huginn \
65 
+        -e HUGINN_DATABASE_PASSWORD=somethingsecret \
66 
+        cantino/huginn
67 
+
68 
+To link to another container named 'postgres':
69 
+
70 
+    docker run --name huginn_postgres \
71 
+        -e POSTGRES_PASSWORD=mysecretpassword \
72 
+        -e POSTGRES_USER=huginn -d postgres
73 
+    docker run --rm --name huginn \
74 
+        --link huginn_postgres:postgres \
75 
+        -p 3000:3000 \
76 
+        -e HUGINN_DATABASE_USERNAME=huginn \
77 
+        -e HUGINN_DATABASE_PASSWORD=mysecretpassword \
78 
+        -e HUGINN_DATABASE_ADAPTER=postgresql \
79 
+        cantino/huginn
80 
+
81 
+The `docker/multi-process` folder also has a `docker-compose.yml` that allows for a sample database formation with a data volume container:
82 
+
83 
+    cd docker/multi-process ; docker-compose up
84 
+
85 
+## Environment Variables
86 
+
87 
+Other Huginn 12factored environment variables of note, as generated and put into the .env file as per Huginn documentation. All variables of the [.env.example](https://github.com/cantino/huginn/blob/master/.env.example) can be used to override the defaults which a read from the current `.env.example`.
88 
+
89 
+For variables in the .env.example that are commented out, the default is to not include that variable in the generated .env file.
90 
+
91 
+## Building on your own
92 
+
93 
+You don't need to do this on your own, because there is an [automated build](https://registry.hub.docker.com/u/cantino/huginn/) for this repository, but if you really want:
94 
+
95 
+    docker build --rm=true --tag={yourname}/huginn .
96 
+
97 
+## Source
98 
+
99 
+The source is [available on GitHub](https://github.com/cantino/huginn/docker/multi-process/).
100 
+
101 
+Please feel free to submit pull requests and/or fork at your leisure.
102 
+
103 
+

docker/docker-compose.yml → docker/multi-process/docker-compose.yml
Преглед на файла


+ 32 - 24
docker/scripts/init
Преглед на файла 

@@ -3,9 +3,6 @@ set -e
3 3
4 4 
 cd /app
5 5
6 
-# Default to the environment variable values set in .env.example
7 
-source /app/.env.example
8 
-
9 6 
 # Cleanup any leftover pid file
10 7 
 if [ -f /app/tmp/pids/server.pid ]; then
11 8 
   rm /app/tmp/pids/server.pid
 
@@ -15,28 +12,42 @@ fi
15 12 
 # requires that the mysql or postgresql containers have exposed
16 13 
 # port 3306 and 5432 respectively.
17 14 
 if [ -n "${MYSQL_PORT_3306_TCP_ADDR}" ]; then
18 
-  HUGINN_DATABASE_ADAPTER=${HUGINN_DATABASE_ADAPTER:-mysql2}
19 
-  HUGINN_DATABASE_HOST=${HUGINN_DATABASE_HOST:-${MYSQL_PORT_3306_TCP_ADDR}}
20 
-  HUGINN_DATABASE_PORT=${HUGINN_DATABASE_PORT:-${MYSQL_PORT_3306_TCP_PORT}}
15 
+  DATABASE_ADAPTER=${DATABASE_ADAPTER:-mysql2}
16 
+  DATABASE_HOST=${DATABASE_HOST:-${MYSQL_PORT_3306_TCP_ADDR}}
17 
+  DATABASE_PORT=${DATABASE_PORT:-${MYSQL_PORT_3306_TCP_PORT}}
18 
+  DATABASE_ENCODING=${DATABASE_ENCODING:-utf8mb4}
21 19 
 elif [ -n "${POSTGRES_PORT_5432_TCP_ADDR}" ]; then
22 
-  HUGINN_DATABASE_ADAPTER=${HUGINN_DATABASE_ADAPTER:-postgresql}
23 
-  HUGINN_DATABASE_HOST=${HUGINN_DATABASE_HOST:-${POSTGRES_PORT_5432_TCP_ADDR}}
24 
-  HUGINN_DATABASE_PORT=${HUGINN_DATABASE_PORT:-${POSTGRES_PORT_5432_TCP_PORT}}
20 
+  DATABASE_ADAPTER=${DATABASE_ADAPTER:-postgresql}
21 
+  DATABASE_HOST=${DATABASE_HOST:-${POSTGRES_PORT_5432_TCP_ADDR}}
22 
+  DATABASE_PORT=${DATABASE_PORT:-${POSTGRES_PORT_5432_TCP_PORT}}
23 
+  DATABASE_ENCODING=utf8
24 
+else
25 
+  START_MYSQL=${START_MYSQL:-true}
25 26 
 fi
26 27
27 
-grep = /app/.env.example | sed -e 's/^#\([^ ]\)/\1/' | grep -v -e '^#' | cut -d= -f1 | \
28 
-  while read var ; do
29 
-    eval "echo \"$var=\\\"\${HUGINN_$var:-\$$var}\\\"\""
28 
+USE_GRAPHVIZ_DOT=${HUGINN_USE_GRAPHVIZ_DOT:-${USE_GRAPHVIZ_DOT:-dot}}
29 
+DATABASE_HOST=${HUGINN_DATABASE_HOST:-${DATABASE_HOST:-localhost}}
30 
+DATABASE_PORT=${HUGINN_DATABASE_PORT:-${DATABASE_PORT}}
31 
+DATABASE_ENCODING=${HUGINN_DATABASE_ENCODING:-${DATABASE_ENCODING}}
32 
+DATABASE_PASSWORD=${HUGINN_DATABASE_PASSWORD:-${DATABASE_PASSWORD:-password}}
33 
+DATABASE_NAME=${HUGINN_DATABASE_NAME:-${DATABASE_NAME:-huginn_production}}
34 
+RAILS_ENV=${HUGINN_RAILS_ENV:-${RAILS_ENV:-production}}
35 
+
36 
+IFS="="
37 
+grep = /app/.env.example | sed -e 's/^#\([^ ]\)/\1/' | grep -v -e '^#' | \
38 
+  while read var value ; do
39 
+    eval "echo \"$var=\${$var:-\${HUGINN_$var-\$value}}\""
30 40 
   done | grep -v -e ^= > /app/.env
31 41
42 
+eval "echo RAILS_ENV=${RAILS_ENV}" >> .env
43 
+eval "echo START_MYSQL=${START_MYSQL}" >> .env
44 
+echo "ON_HEROKU=true" >> .env
45 
+echo "RAILS_SERVE_STATIC_FILES=true" >> .env
46 
+
32 47 
 chmod ugo+r /app/.env
33 48 
 source /app/.env
34 49 
 sudo -u huginn -H bundle install --deployment --without test
35 50
36 
-DATABASE_HOST=${HUGINN_DATABASE_HOST:-${DATABASE_HOST:-localhost}}
37 
-DATABASE_ENCODING=${HUGINN_DATABASE_ENCODING:-${DATABASE_ENCODING}}
38 
-USE_GRAPHVIZ_DOT=${HUGINN_USE_GRAPHVIZ_DOT:-${USE_GRAPHVIZ_DOT}}
39 
-
40 51 
 # use default port number if it is still not set
41 52 
 case "${DATABASE_ADAPTER}" in
42 53 
   mysql2) DATABASE_PORT=${DATABASE_PORT:-3306} ;;
 
@@ -71,7 +82,8 @@ source /app/.env
71 82 
 echo DATABASE_HOST=\${DATABASE_HOST}
72 83
73 84 
 # start mysql server if \${DATABASE_HOST} is the .env.example default
74 
-if [ "\${DATABASE_HOST}" = "your-domain-here.com" -o "\${DATABASE_HOST}" = "" ]; then
85 
+if [ "\${START_MYSQL}" = "true" ]; then
86 
+  echo "DATABASE_SOCKET=/var/run/mysqld/mysqld.sock" >> .env
75 87 
   if [ "\${DATABASE_ADAPTER}" = "postgresql" ]; then
76 88 
     echo "DATABASE_ADAPTER 'postgresql' is not supported internally. Please provide DATABASE_HOST."
77 89 
     exit 1
 
@@ -106,7 +118,7 @@ EOF
106 118
107 119 
   # wait for mysql server to start (max 120 seconds)
108 120 
   timeout=120
109 
-  while ! mysqladmin -u\${DATABASE_USERNAME:-root} \${DATABASE_PASSWORD:+-p\$DATABASE_PASSWORD} status >/dev/null 2>&1
121 
+  while ! mysqladmin -u root status >/dev/null 2>&1
110 122 
   do
111 123 
     (( timeout = timeout - 1 ))
112 124 
     if [ \$timeout -eq 0 ]; then
 
@@ -118,9 +130,7 @@ EOF
118 130 
   done
119 131
120 132 
   if ! echo "USE \${DATABASE_NAME}" | mysql -u\${DATABASE_USERNAME:-root} \${DATABASE_PASSWORD:+-p\$DATABASE_PASSWORD} >/dev/null 2>&1; then
121 
-    DB_INIT="yes"
122 
-    echo "CREATE DATABASE IF NOT EXISTS \\\`\${DATABASE_NAME}\\\` DEFAULT CHARACTER SET \\\`utf8\\\` COLLATE \\\`utf8_unicode_ci\\\`;" | mysql -uroot \${DATABASE_PASSWORD:+-p\$DATABASE_PASSWORD}
123 
-    echo "GRANT SELECT, LOCK TABLES, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER ON \\\`${DATABASE_NAME}\\\`.* TO 'root'@'localhost';" | mysql -u\${DATABASE_USERNAME:-root} \${DATABASE_PASSWORD:+-p\$DATABASE_PASSWORD}
133 
+    echo "SET PASSWORD FOR 'root'@'localhost' = PASSWORD('\${DATABASE_PASSWORD:\$DATABASE_PASSWORD}');" | mysql -u root
124 134 
   fi
125 135 
 fi
126 136 
 supervisorctl start foreman >/dev/null
 
@@ -165,9 +175,7 @@ if [ -n "\${DATABASE_INITIAL_CONNECT_MAX_RETRIES}" ]; then
165 175 
 fi
166 176
167 177 
 # We may need to try and create a database
168 
-if [ -n "\${CREATE_DB}" ]; then
169 
- sudo -u huginn -EH bundle exec rake db:create
170 
-fi
178 
+sudo -u huginn -EH bundle exec rake db:create
171 179
172 180 
 # Assuming we have a created database, run the migrations and seed it idempotently.
173 181 
 if [ -z "\${DO_NOT_MIGRATE}" ]; then

+ 9 - 0
docker/multi-process/scripts/standalone-packages
Преглед на файла 

@@ -0,0 +1,9 @@
1 
+DEBIAN_FRONTEND=noninteractive
2 
+apt-get update
3 
+apt-get install -y python2.7 python-docutils mysql-server \
4 
+                   supervisor python-pip && \
5 
+pip install supervisor-stdout
6 
+rm -rf /var/lib/apt/lists/*
7 
+rm -rf /usr/share/doc/
8 
+rm -rf /usr/share/man/
9 
+rm -rf /usr/share/locale/

+ 0 - 38
docker/scripts/setup
Преглед на файла 

@@ -1,38 +0,0 @@
1 
-#!/bin/bash
2 
-set -e
3 
-
4 
-locale-gen en_US.UTF-8
5 
-
6 
-# Shallow clone the huginn project repo
7 
-git clone --depth 1 https://github.com/cantino/huginn /app
8 
-
9 
-cd /app
10 
-
11 
-# add a huginn group and user
12 
-adduser --group huginn
13 
-adduser --disabled-login --ingroup huginn --gecos 'Huginn' --no-create-home --home /app huginn
14 
-adduser huginn sudo
15 
-passwd -d huginn
16 
-
17 
-# Change the ownership to huginn
18 
-chown -R huginn:huginn /app
19 
-
20 
-# create required tmp and log directories
21 
-sudo -u huginn -H mkdir -p tmp/pids tmp/cache tmp/sockets log
22 
-chmod -R u+rwX log tmp
23 
-
24 
-# install gems required by Huginn, use local cache if available
25 
-if [ -d "/scripts/cache" ]; then
26 
-  mv /scripts/cache vendor/
27 
-  chown -R huginn:huginn vendor/cache
28 
-fi
29 
-sudo -u huginn -H cp .env.example .env
30 
-sudo -u huginn -H LC_ALL="en_US.UTF-8" LANG="en_US.UTF-8" ON_HEROKU=true bundle install --deployment --without test
31 
-sudo -u huginn -H rm .env
32 
-
33 
-# silence setlocale message (THANKS DEBIAN!)
34 
-cat > /etc/default/locale <<EOF
35 
-LC_ALL=en_US.UTF-8
36 
-LANG=en_US.UTF-8
37 
-EOF
38 
-

+ 16 - 0
docker/single-process/Dockerfile
Преглед на файла 

@@ -0,0 +1,16 @@
1 
+FROM ubuntu:14.04
2 
+MAINTAINER Dominik Sander
3 
+
4 
+ADD scripts/prepare /scripts/prepare
5 
+RUN /scripts/prepare
6 
+
7 
+ADD scripts/setup /scripts/setup
8 
+RUN /scripts/setup
9 
+
10 
+WORKDIR /app
11 
+
12 
+ADD scripts/init /scripts/init
13 
+
14 
+EXPOSE 3000
15 
+
16 
+CMD ["/scripts/init"]

+ 94 - 0
docker/single-process/README.md
Преглед на файла 

@@ -0,0 +1,94 @@
1 
+Docker image for Huginn using the production environment and separate container for every process
2 
+=================================================
3 
+
4 
+This image runs a linkable [Huginn](https://github.com/cantino/huginn) instance.
5 
+
6 
+It was inspired by the [official docker container for huginn](https://registry.hub.docker.com/u/cantino/huginn)
7 
+
8 
+The scripts/init script generates a .env file containing the variables as passed as per normal Huginn documentation.
9 
+The same environment variables that would be used for Heroku PaaS deployment are used by this script.
10 
+
11 
+The scripts/init script is aware of mysql and postgres linked containers through the environment variables:
12 
+
13 
+    MYSQL_PORT_3306_TCP_ADDR
14 
+    MYSQL_PORT_3306_TCP_PORT
15 
+
16 
+and
17 
+
18 
+    POSTGRESQL_PORT_5432_TCP_ADDR
19 
+    POSTGRESQL_PORT_5432_TCP_PORT
20 
+
21 
+Its recommended to use an image that allows you to create a database via environmental variables at docker run, like `postgresql` or `mysql`, so the db is populated when this script runs.
22 
+
23 
+Additionally, the database variables may be overridden from the above as per the standard Huginn documentation:
24 
+
25 
+    DATABASE_ADAPTER #(must be either 'postgresql' or 'mysql2')
26 
+    DATABASE_HOST
27 
+    DATABASE_PORT
28 
+
29 
+This script will run database migrations (rake db:migrate) which should be idempotent.
30 
+
31 
+It will also seed the database (rake db:seed) unless this is defined:
32 
+
33 
+    DO_NOT_SEED
34 
+
35 
+This same seeding initially defines the "admin" user with a default password of "password" as per the standard Huginn documentation.
36 
+
37 
+If you do not wish to have the default 6 agents, you will want to set the above environment variable after your initially deploy, otherwise they will be added automatically the next time a container pointing at the database is spun up.
38 
+
39 
+The CMD launches Huginn via the scripts/init script. This may become the ENTRYPOINT later.  It does take under a minute for Huginn to come up.  Use environmental variables that match your DB's creds to ensure it works.
40 
+
41 
+## Usage
42 
+
43 
+Simple startup using docker compose (you need to daemonize with `-d` to persist the data):
44 
+
45 
+    cd docker/single-process
46 
+    docker-compose up
47 
+
48 
+or if you like to use PostgreSQL:
49 
+
50 
+    docker-compose -f postgresql.yml up
51 
+
52 
+Manual startup and linking to a MySQL container:
53 
+
54 
+    docker run --name huginn_mysql \
55 
+        -e MYSQL_DATABASE=huginn \
56 
+        -e MYSQL_USER=huginn \
57 
+        -e MYSQL_PASSWORD=somethingsecret \
58 
+        -e MYSQL_ROOT_PASSWORD=somethingevenmoresecret \
59 
+        mysql
60 
+
61 
+    docker run --name huginn_web \
62 
+        --link huginn_mysql:mysql \
63 
+        -p 3000:3000 \
64 
+        -e DATABASE_NAME=huginn \
65 
+        -e DATABASE_USERNAME=huginn \
66 
+        -e DATABASE_PASSWORD=somethingsecret \
67 
+        cantino/huginn-single-process
68 
+
69 
+    docker run --name huginn_threaded \
70 
+        --link huginn_mysql:mysql \
71 
+        -e DATABASE_NAME=huginn \
72 
+        -e DATABASE_USERNAME=huginn \
73 
+        -e DATABASE_PASSWORD=somethingsecret \
74 
+        cantino/huginn-single-process /scripts/init bin/threaded.rb
75 
+
76 
+## Environment Variables
77 
+
78 
+Other Huginn 12factored environment variables of note, as generated and put into the .env file as per Huginn documentation. All variables of the [.env.example](https://github.com/cantino/huginn/blob/master/.env.example) can be used to override the defaults which a read from the current `.env.example`.
79 
+
80 
+For variables in the .env.example that are commented out, the default is to not include that variable in the generated .env file.
81 
+
82 
+## Building on your own
83 
+
84 
+You don't need to do this on your own, but if you really want:
85 
+
86 
+    docker build --rm=true --tag={yourname}/huginn .
87 
+
88 
+## Source
89 
+
90 
+The source is [available on GitHub](https://github.com/cantino/huginn/docker/single-process/).
91 
+
92 
+Please feel free to submit pull requests and/or fork at your leisure.
93 
+
94 
+

+ 35 - 0
docker/single-process/develop.yml
Преглед на файла 

@@ -0,0 +1,35 @@
1 
+mysqldata:
2 
+  image: mysql
3 
+  command: /bin/true
4 
+
5 
+mysql:
6 
+  image: mysql
7 
+  volumes_from:
8 
+    - mysqldata
9 
+  environment:
10 
+    MYSQL_ROOT_PASSWORD: myrootpassword
11 
+    MYSQL_DATABASE: huginn
12 
+    MYSQL_USER: huginn
13 
+    MYSQL_PASSWORD: myhuginnpassword
14 
+
15 
+huginn_web:
16 
+  build: .
17 
+  restart: always
18 
+  extends:
19 
+    file: environment.yml
20 
+    service: huginn_base
21 
+  ports:
22 
+    - 3000:3000
23 
+  links:
24 
+    - mysql
25 
+
26 
+huginn_threaded:
27 
+  build: .
28 
+  restart: always
29 
+  extends:
30 
+    file: environment.yml
31 
+    service: huginn_base
32 
+  links:
33 
+    - mysql
34 
+  command: /scripts/init bin/threaded.rb
35 
+

+ 35 - 0
docker/single-process/docker-compose.yml
Преглед на файла 

@@ -0,0 +1,35 @@
1 
+mysqldata:
2 
+  image: mysql
3 
+  command: /bin/true
4 
+
5 
+mysql:
6 
+  image: mysql
7 
+  volumes_from:
8 
+    - mysqldata
9 
+  environment:
10 
+    MYSQL_ROOT_PASSWORD: myrootpassword
11 
+    MYSQL_DATABASE: huginn
12 
+    MYSQL_USER: huginn
13 
+    MYSQL_PASSWORD: myhuginnpassword
14 
+
15 
+huginn_web:
16 
+  image: cantino/huginn-single-process
17 
+  restart: always
18 
+  extends:
19 
+    file: environment.yml
20 
+    service: huginn_base
21 
+  ports:
22 
+    - 3000:3000
23 
+  links:
24 
+    - mysql
25 
+
26 
+huginn_threaded:
27 
+  image: cantino/huginn-single-process
28 
+  restart: always
29 
+  extends:
30 
+    file: environment.yml
31 
+    service: huginn_base
32 
+  links:
33 
+    - mysql
34 
+  command: /scripts/init bin/threaded.rb
35 
+

+ 7 - 0
docker/single-process/environment.yml
Преглед на файла 

@@ -0,0 +1,7 @@
1 
+huginn_base:
2 
+  environment:
3 
+    DATABASE_ADAPTER: mysql2
4 
+    DATABASE_NAME: huginn
5 
+    DATABASE_USERNAME: huginn
6 
+    DATABASE_PASSWORD: myhuginnpassword
7 
+    APP_SECRET_TOKEN: 3bd139f9186b31a85336bb89cd1a1337078921134b2f48e022fd09c234d764d3e19b018b2ab789c6e0e04a1ac9e3365116368049660234c2038dc9990513d49c

+ 72 - 0
docker/single-process/postgresql.yml
Преглед на файла 

@@ -0,0 +1,72 @@
1 
+postgresdata:
2 
+  image: postgres
3 
+  command: /bin/true
4 
+
5 
+postgres:
6 
+  image: postgres
7 
+  volumes_from:
8 
+    - postgresdata
9 
+  environment:
10 
+    POSTGRES_PASSWORD: myhuginnpassword
11 
+    POSTGRES_USER: huginn
12 
+
13 
+huginn_web:
14 
+  image: cantino/huginn-single-process
15 
+  restart: always
16 
+  extends:
17 
+    file: environment.yml
18 
+    service: huginn_base
19 
+  environment:
20 
+    DATABASE_ADAPTER: postgresql
21 
+  ports:
22 
+    - 3000:3000
23 
+  links:
24 
+    - postgres
25 
+
26 
+huginn_threaded:
27 
+  image: cantino/huginn-single-process
28 
+  restart: always
29 
+  extends:
30 
+    file: environment.yml
31 
+    service: huginn_base
32 
+  environment:
33 
+    DATABASE_ADAPTER: postgresql
34 
+  links:
35 
+    - postgres
36 
+  command: /scripts/init bin/threaded.rb
37 
+
38 
+# huginn_schedule:
39 
+#   image: cantino/huginn-single-process
40 
+#   extends:
41 
+#     file: environment.yml
42 
+#     service: huginn_base
43 
+#   environment:
44 
+#     DATABASE_ADAPTER: postgresql
45 
+#   links:
46 
+#     - postgres
47 
+#   command: /scripts/init bin/schedule.rb
48 
+
49 
+
50 
+# huginn_twitter_stream:
51 
+#   image: cantino/huginn-single-process
52 
+#   extends:
53 
+#     file: environment.yml
54 
+#     service: huginn_base
55 
+#   environment:
56 
+#     DATABASE_ADAPTER: postgresql
57 
+#   links:
58 
+#     - postgres
59 
+#   command: /scripts/init bin/twitter_stream.rb
60 
+
61 
+
62 
+# huginn_dj1:
63 
+#   image: cantino/huginn-single-process
64 
+#   extends:
65 
+#     file: environment.yml
66 
+#     service: huginn_base
67 
+#   environment:
68 
+#     DATABASE_ADAPTER: postgresql
69 
+#   links:
70 
+#     - postgres
71 
+#   command: /scripts/init script/delayed_job run
72 
+

+ 57 - 0
docker/single-process/scripts/init
Преглед на файла 

@@ -0,0 +1,57 @@
1 
+#!/bin/bash
2 
+set -e
3 
+
4 
+cd /app
5 
+
6 
+# Configure database based on linked container
7 
+if [ -n "${MYSQL_PORT_3306_TCP_ADDR}" ]; then
8 
+  DATABASE_ADAPTER=${DATABASE_ADAPTER:-mysql2}
9 
+  DATABASE_HOST=${DATABASE_HOST:-${MYSQL_PORT_3306_TCP_ADDR}}
10 
+  DATABASE_PORT=${DATABASE_PORT:-${MYSQL_PORT_3306_TCP_PORT}}
11 
+  DATABASE_ENCODING=${DATABASE_ENCODING:-utf8mb4}
12 
+elif [ -n "${POSTGRES_PORT_5432_TCP_ADDR}" ]; then
13 
+  DATABASE_ADAPTER=${DATABASE_ADAPTER:-postgresql}
14 
+  DATABASE_HOST=${DATABASE_HOST:-${POSTGRES_PORT_5432_TCP_ADDR}}
15 
+  DATABASE_PORT=${DATABASE_PORT:-${POSTGRES_PORT_5432_TCP_PORT}}
16 
+  DATABASE_ENCODING=utf8
17 
+fi
18 
+
19 
+USE_GRAPHVIZ_DOT=${USE_GRAPHVIZ_DOT:-${USE_GRAPHVIZ_DOT:-dot}}
20 
+
21 
+# Default to the environment variable values set in .env.example
22 
+IFS="="
23 
+grep = /app/.env.example | sed -e 's/^#\([^ ]\)/\1/' | grep -v -e '^#' | \
24 
+  while read var value ; do
25 
+    eval "echo \"$var=\${$var:-\${HUGINN_$var-\$value}}\""
26 
+  done | grep -v -e ^= > /app/.env
27 
+
28 
+eval "echo PORT=${PORT:-${PORT:-3000}}" >> .env
29 
+eval "echo RAILS_ENV=${RAILS_ENV:-${RAILS_ENV:-production}}" >> .env
30 
+eval "echo ON_HEROKU=true" >> .env
31 
+eval "echo RAILS_SERVE_STATIC_FILES=true" >> .env
32 
+
33 
+chmod ugo+r /app/.env
34 
+source /app/.env
35 
+
36 
+# use default port number if it is still not set
37 
+case "${DATABASE_ADAPTER}" in
38 
+  mysql2) DATABASE_PORT=${DATABASE_PORT:-3306} ;;
39 
+  postgresql) DATABASE_PORT=${DATABASE_PORT:-5432} ;;
40 
+  *) echo "Unsupported database adapter. Available adapters are mysql2, and postgresql." && exit 1 ;;
41 
+esac
42 
+
43 
+sudo -u huginn -H bundle install --without test development --path vendor/bundle
44 
+
45 
+if [ -z $1 ]; then
46 
+  sudo -u huginn -H bundle exec rake db:create db:migrate RAILS_ENV=${RAILS_ENV}
47 
+fi
48 
+
49 
+if [[ -z "${DO_NOT_SEED}" && -z $1 ]]; then
50 
+  sudo -u huginn -H bundle exec rake db:seed RAILS_ENV=${RAILS_ENV}
51 
+fi
52 
+
53 
+if [ -z $1 ]; then
54 
+  exec sudo -u huginn -H bundle exec unicorn -c config/unicorn.rb
55 
+else
56 
+  exec sudo -u huginn -H bundle exec rails runner "$@" RAILS_ENV=${RAILS_ENV}
57 
+fi

+ 40 - 0
docker/single-process/scripts/prepare
Преглед на файла 

@@ -0,0 +1,40 @@
1 
+#!/bin/bash
2 
+set -e
3 
+
4 
+cat > /etc/dpkg/dpkg.cfg.d/01_nodoc <<EOF
5 
+# Delete locales
6 
+path-exclude=/usr/share/locale/*
7 
+
8 
+# Delete man pages
9 
+path-exclude=/usr/share/man/*
10 
+
11 
+# Delete docs
12 
+path-exclude=/usr/share/doc/*
13 
+path-include=/usr/share/doc/*/copyright
14 
+EOF
15 
+
16 
+export LC_ALL=C
17 
+export DEBIAN_FRONTEND=noninteractive
18 
+minimal_apt_get_install='apt-get install -y --no-install-recommends'
19 
+
20 
+apt-get update
21 
+apt-get dist-upgrade -y --no-install-recommends
22 
+$minimal_apt_get_install software-properties-common
23 
+add-apt-repository -y ppa:brightbox/ruby-ng
24 
+apt-get update
25 
+$minimal_apt_get_install build-essential checkinstall git-core \
26 
+  zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev \
27 
+  libncurses5-dev libffi-dev libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev \
28 
+  graphviz libgraphviz-dev \
29 
+  libmysqlclient-dev libpq-dev libsqlite3-dev \
30 
+  ruby2.2 ruby2.2-dev
31 
+locale-gen en_US.UTF-8
32 
+update-locale LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8
33 
+gem install --no-ri --no-rdoc bundler
34 
+
35 
+apt-get purge -y python3* rsyslog rsync manpages
36 
+rm -rf /var/lib/apt/lists/*
37 
+rm -rf /usr/share/doc/
38 
+rm -rf /usr/share/man/
39 
+rm -rf /usr/share/locale/
40 
+rm -rf /var/log/*

+ 36 - 0
docker/single-process/scripts/setup
Преглед на файла 

@@ -0,0 +1,36 @@
1 
+#!/bin/bash
2 
+set -e
3 
+
4 
+# add a huginn group and user
5 
+adduser --group huginn
6 
+adduser --disabled-login --ingroup huginn --gecos 'Huginn' --no-create-home --home /app huginn
7 
+passwd -d huginn
8 
+
9 
+# Shallow clone the huginn project repo
10 
+git clone --depth 1 https://github.com/cantino/huginn /app
11 
+
12 
+# Change the ownership to huginn
13 
+chown -R huginn:huginn /app
14 
+
15 
+cd app
16 
+
17 
+# create required tmp and log directories
18 
+sudo -u huginn -H mkdir -p tmp/pids tmp/cache tmp/sockets log
19 
+chmod -R u+rwX log tmp
20 
+
21 
+export LC_ALL=en_US.UTF-8
22 
+
23 
+# HACK: We need a database connection to precompile the assets, use sqlite for that
24 
+echo "gem 'sqlite3', '~> 1.3.11'" >> Gemfile
25 
+sudo -u huginn -H RAILS_ENV=production APP_SECRET_TOKEN=secret DATABASE_ADAPTER=sqlite3 ON_HEROKU=true bundle install --without test development --path vendor/bundle -j 4
26 
+sudo -u huginn -H RAILS_ENV=production APP_SECRET_TOKEN=secret DATABASE_ADAPTER=sqlite3 ON_HEROKU=true bundle exec rake assets:clean assets:precompile
27 
+git checkout Gemfile
28 
+
29 
+# Bundle again to get rid of the sqlite3 gem
30 
+sudo -u huginn -H ON_HEROKU=true DATABASE_ADAPTER=noop bundle install --without test development --path vendor/bundle
31 
+
32 
+# Configure the unicorn server
33 
+mv config/unicorn.rb.example config/unicorn.rb
34 
+sed -ri 's/^listen .*$/listen ENV["PORT"]/' config/unicorn.rb
35 
+sed -ri 's/^stderr_path.*$//' config/unicorn.rb
36 
+sed -ri 's/^stdout_path.*$//' config/unicorn.rb