archive/bramw_baserow
1
0
Fork 0
mirror of https://gitlab.com/bramw/baserow.git synced 2025-04-17 18:32:35 +00:00
Code Issues Projects Releases Wiki Activity

Resolve "Add a guide to deploy on Heroku"

Browse source
This commit is contained in:
Bram Wiepjes 2021-07-13 19:37:37 +00:00
parent def00370c5
commit d2d7fde595
12 changed files with 495 additions and 10 deletions

26
README.md
View file

@ -1,6 +1,10 @@
# Baserow
Open source no-code database tool and Airtable alternative.
Open source no-code database tool and Airtable alternative. Join our community on
Gitter https://gitter.im/bramw-baserow/community.
[![Deploy to Heroku](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy?template=https://github.com/bram2w/baserow/tree/master)
*Beta*
**We're hiring** remote developers! More information at
https://baserow.io/jobs/experienced-full-stack-developer.
@ -26,13 +30,11 @@ you the powers of a developer without leaving your browser.
* Runs in your browser.
* Alternative to Airtable.
## Become a sponsor
## Self host
If you would like to get new features faster then you might want to consider becoming
a sponsor. By becoming a sponsor we can spend more time on Baserow which means faster
development.
[Become a GitHub Sponsor](https://github.com/sponsors/bram2w)
* [**Heroku**: Easily install and scale up on Heroku by using the template.](./docs/guides/installation/install-on-heroku.md)
* [**Cloudron**: Install and update Baserow on your own Cloudron server.](./docs/guides/installation/install-on-cloudron.md)
* [**Ubuntu**: Manually install Baserow step by step on a clean Ubuntu server. **not recommended**](./docs/guides/installation/install-on-ubuntu.md)
## Try out Baserow on your machine
@ -90,9 +92,17 @@ or [here](./docs/index.md) inside the repository. The API docs can be found here
https://api.baserow.io/api/redoc/ or if you are looking for the OpenAPI schema here
https://api.baserow.io/api/schema.json.
## Become a sponsor
If you would like to get new features faster, then you might want to consider becoming
a sponsor. By becoming a sponsor we can spend more time on Baserow which means faster
development.
[Become a GitHub Sponsor](https://github.com/sponsors/bram2w)
## Meta
Created by Bram Wiepjes (Baserow) - bram@baserow.io.
Created by Baserow B.V. - bram@baserow.io.
Distributes under the MIT license. See `LICENSE` for more information.

36
app.json Normal file
View file

@ -0,0 +1,36 @@
{
"name": "Baserow",
"description": "Open source online no-code database and Airtable alternative. More information about persistent files on S3, scaling and installing a custom domain can be found on https://baserow.io/docs/guides/installation/install-on-heroku. Note that it might take a couple of minutes to deploy the app, so please be patient.",
"repository": "https://gitlab.com/bramw/baserow",
"keywords": ["baserow", "no-code", "database", "airtable"],
"logo": "https://baserow.io/img/favicon_192.png",
"success_url": "/",
"addons": [
"heroku-postgresql:hobby-dev",
"heroku-redis:hobby-dev",
"mailgun:starter"
],
"formation": {
"web": {
"quantity": 1,
"size": "free"
}
},
"stack": "container",
"env": {
"SECRET_KEY": {
"description": "A unique string that is used to generate secrets.",
"generator": "secret"
},
"BASEROW_PUBLIC_URL": {
"description": "The public URL of your Heroku Baserow app. Don't forget to replace YOUR_APP_NAME with the name of your app.",
"value": "https://YOUR_APP_NAME.herokuapp.com",
"required": true
},
"BASEROW_AMOUNT_OF_WORKERS": {
"description": "The amount of workers per dyno. This value must be `1` if you want to run the free plan.",
"value": "1",
"required": true
}
}
}

5
backend/src/baserow/config/settings/base.py
View file

@ -74,8 +74,9 @@ REDIS_PORT = os.getenv("REDIS_PORT", "6379")
REDIS_USERNAME = os.getenv("REDIS_USER", "")
REDIS_PASSWORD = os.getenv("REDIS_PASSWORD", "")
REDIS_PROTOCOL = os.getenv("REDIS_PROTOCOL", "redis")
REDIS_URL = (
f"{REDIS_PROTOCOL}://{REDIS_USERNAME}:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}/0"
REDIS_URL = os.getenv(
"REDIS_URL",
f"{REDIS_PROTOCOL}://{REDIS_USERNAME}:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}/0",
)
CELERY_BROKER_URL = REDIS_URL

1
changelog.md
View file

@ -2,6 +2,7 @@
## Unreleased
* Added a Heroku template and one click deploy button.
* Fixed bug preventing the deletion of rows with a blank single select primary field
* Fixed error in trash cleanup job when deleting multiple rows and a field from the same
table at once.

34
deploy/heroku/README.md Normal file
View file

@ -0,0 +1,34 @@
## Testing
In order to test your changes, you need to have the Heroku command line installed on
your local machine. Next, you can use the Heroku command line to create an app,
manually install the `addons` and `labs` listed in the app.json file at the root of
this repo. In the example below we assume you are at the root of the Baserow repo and
we are going install a heroku app named `baserow-test-app`, this can of course be named
differently.
```
$ heroku apps:create baserow-test-app
$ heroku stack:set -a baserow-test-app container
# We need to add all the addons listed in the app.json file.
$ heroku addons:create -a baserow-test-app heroku-postgresql:hobby-dev
$ heroku addons:create -a baserow-test-app heroku-redis:hobby-dev
$ heroku addons:create -a baserow-test-app mailgun:starter
# We need to add all the labs listed in the app.json file.
$ heroku labs:enable -a baserow-test-app runtime-dyno-metadata
# Finally we need to set all the environment variables listed in the app.json file.
$ heroku config:set -a baserow-test-app SECRET_KEY=REPLACE_WITH_SECRET_VALUE
$ heroku config:set -a baserow-test-app BASEROW_PUBLIC_URL=
$ heroku config:set -a baserow-test-app BASEROW_AMOUNT_OF_WORKERS=1
```
Now that we have replicated the setup of the app.json, we can deploy the application
by pushing to the heroku remote repository.
```
$ git remote add heroku https://git.heroku.com/baserow-test-app.git
$ git push heroku YOUR_CURRENT_BRANCH:master
```

7
deploy/heroku/entry.sh Normal file
View file

@ -0,0 +1,7 @@
#!/bin/bash
set -eu
sed -i 's/PORT/'"$PORT"'/g' /baserow/nginx.conf
export BASEROW_PUBLIC_URL=${BASEROW_PUBLIC_URL:-https://$HEROKU_APP_NAME.herokuapp.com}
export REDIS_URL=${REDIS_TLS_URL:-$REDIS_URL}

67
deploy/heroku/nginx.conf Normal file
View file

@ -0,0 +1,67 @@
events {
worker_connections 768;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
gzip on;
client_body_temp_path /run/client_body;
proxy_temp_path /run/proxy_temp;
fastcgi_temp_path /run/fastcgi_temp;
scgi_temp_path /run/scgi_temp;
uwsgi_temp_path /run/uwsgi_temp;
server {
access_log /dev/stdout;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
gzip on;
gzip_disable "msie6";
listen PORT;
proxy_read_timeout 1800s;
client_max_body_size 0; # avoid HTTP 413 for large image uploads
# required to avoid HTTP 411: see Issue #1486 (https://github.com/dotcloud/docker/issues/1486)
chunked_transfer_encoding on;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_pass_request_headers on;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
location ~ ^/(api|ws)/ {
proxy_pass http://127.0.0.1:8000;
proxy_pass_request_headers on;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
location /media/ {
root /baserow;
add_header Content-Disposition "attachment";
}
}
}

62
deploy/heroku/settings.py Normal file
View file

@ -0,0 +1,62 @@
from baserow.config.settings.base import *
import os
import ssl
import dj_database_url
INSTALLED_APPS = INSTALLED_APPS + ["storages"]
MEDIA_ROOT = "/baserow/media"
MJML_BACKEND_MODE = "cmd"
MJML_EXEC_CMD = "mjml"
FROM_EMAIL = f"no-reply@{os.environ['MAILGUN_DOMAIN']}"
CELERY_EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
EMAIL_USE_TLS = False
EMAIL_HOST = os.environ["MAILGUN_SMTP_SERVER"]
EMAIL_PORT = os.environ["MAILGUN_SMTP_PORT"]
EMAIL_HOST_USER = os.environ["MAILGUN_SMTP_LOGIN"]
EMAIL_HOST_PASSWORD = os.environ["MAILGUN_SMTP_PASSWORD"]
# We need to set the certificate check to None, otherwise it is not compatible with the
# `heroku-redis:hobby-dev` addon. The URL generated by that addon is over a secured
# connection with a self signed certificate. The redis broker could fail if the
# certificate can't be verified.
CELERY_REDBEAT_REDIS_USE_SSL = {"ssl_cert_reqs": ssl.CERT_NONE}
ssl_context = ssl.SSLContext()
ssl_context.verify_mode = ssl.CERT_NONE
CHANNELS_REDIS_HOST = {"address": REDIS_URL, "ssl": ssl_context}
CHANNEL_LAYERS["default"]["CONFIG"]["hosts"] = [CHANNELS_REDIS_HOST]
# Set the limit of the connection pool based on the amount of workers that must be
# started with a limit of 10, which is the default value. This is needed because the
# `heroku-redis:hobby-dev` doesn't accept more than 20 connections.
CELERY_BROKER_POOL_LIMIT = min(4 * int(os.getenv("BASEROW_AMOUNT_OF_WORKERS", "1")), 10)
CELERY_REDIS_MAX_CONNECTIONS = min(
4 * int(os.getenv("BASEROW_AMOUNT_OF_WORKERS", "1")), 10
)
DATABASES = {
"default": dj_database_url.parse(os.environ["DATABASE_URL"], conn_max_age=600)
}
if os.getenv("AWS_ACCESS_KEY_ID", "") != "":
DEFAULT_FILE_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"
AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")
AWS_STORAGE_BUCKET_NAME = os.getenv("AWS_STORAGE_BUCKET_NAME")
AWS_S3_OBJECT_PARAMETERS = {
"CacheControl": "max-age=86400",
"ContentDisposition": "attachment",
}
AWS_S3_FILE_OVERWRITE = True
AWS_DEFAULT_ACL = "public-read"
if os.getenv("AWS_S3_REGION_NAME", "") != "":
AWS_S3_REGION_NAME = os.getenv("AWS_S3_REGION_NAME")
if os.getenv("AWS_S3_ENDPOINT_URL", "") != "":
AWS_S3_ENDPOINT_URL = os.getenv("AWS_S3_ENDPOINT_URL")
if os.getenv("AWS_S3_CUSTOM_DOMAIN", "") != "":
AWS_S3_CUSTOM_DOMAIN = os.getenv("AWS_S3_CUSTOM_DOMAIN")

70
deploy/heroku/supervisor.conf Normal file
View file

@ -0,0 +1,70 @@
[supervisord]
nodaemon = true
logfile=/dev/null
logfile_maxbytes=0
environment =
DJANGO_SETTINGS_MODULE='heroku.settings',
PRIVATE_BACKEND_URL='http://localhost:8000',
PUBLIC_WEB_FRONTEND_URL='%(ENV_BASEROW_PUBLIC_URL)s',
PUBLIC_BACKEND_URL='%(ENV_BASEROW_PUBLIC_URL)s'
[program:gunicorn]
directory=/baserow/baserow
command=/baserow/env/bin/gunicorn -w %(ENV_BASEROW_AMOUNT_OF_WORKERS)s -b 127.0.0.1:8000 -k uvicorn.workers.UvicornWorker baserow.config.asgi:application --log-level=debug
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stdout
stderr_logfile_maxbytes=0
autostart=true
autorestart=true
[program:worker]
directory=/baserow/baserow
command=/baserow/env/bin/celery -A baserow worker --concurrency %(ENV_BASEROW_AMOUNT_OF_WORKERS)s --without-heartbeat --without-gossip --without-mingle -l INFO
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stdout
stderr_logfile_maxbytes=0
autostart=true
autorestart=true
[program:exportworker]
directory=/baserow/baserow
command=/baserow/env/bin/celery -A baserow worker --concurrency %(ENV_BASEROW_AMOUNT_OF_WORKERS)s --without-heartbeat --without-gossip --without-mingle -l INFO -Q export
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stdout
stderr_logfile_maxbytes=0
autostart=true
sautorestart=true
[program:beatworker]
directory=/baserow/baserow
command=/baserow/env/bin/celery -A baserow beat -l INFO -S redbeat.RedBeatScheduler
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stdout
stderr_logfile_maxbytes=0
autostart=true
autorestart=true
[program:nuxt]
directory=/baserow/baserow/web-frontend
command=sh -c './node_modules/.bin/nuxt start --hostname 127.0.0.1 --port 3000 --config-file ./config/nuxt.config.local.js'
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stdout
stderr_logfile_maxbytes=0
autostart=true
autorestart=true
[program:nginx]
directory=/tmp
command=/usr/sbin/nginx -c /baserow/nginx.conf -g "daemon off;"
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stdout
stderr_logfile_maxbytes=0
autostart=true
autorestart=true
numprocs=1

135
docs/guides/installation/install-on-heroku.md Normal file
View file

@ -0,0 +1,135 @@
# Installation on Heroku
> The Heroku template and one click to deploy button are currently in beta.
Heroku is a platform as a service that enables developer to build, run and operate
applications entirely in the cloud. We have created a template that allows you to
easily install Baserow on the free plan of Heroku. You can also scale up by tweaking
some settings
## The template
[![Deploy to Heroku](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy?template=https://github.com/bram2w/baserow/tree/master)
*Beta*
The button above can be used to install Baserow on Heroku with one click. Click on it
and follow the steps on heroku.com to proceed. After the installation, you can reach
Baserow on the URL provided by Heroku. Everything installed via the template runs on
the free plan of Heroku by default.
## Could not connect to the API server error
If you are getting a "Could not connect to the API server." error when logging in or
creating an account, then you most likely need to update the `BASEROW_PUBLIC_URL`
config var. You can do so by going to the settings page in your Heroku app dashboard,
click on "Reveal Config Vars", find the `BASEROW_PUBLIC_URL` and change the value to
`https://YOUR_APP_NAME.herokuapp.com`. Don't forget to replace `YOUR_APP_NAME` with the
name of your app.
## Persistent S3 file storage
By default, the uploaded files are stored inside the dyno for demo purposes. This means
that everytime your dyno restarts, you will lose all your uploaded files. Your files
can optionally be stored inside an S3 bucket. To do so, you need to add a couple of
config vars to the settings. Go to the Settings page and click on "Reveal Config Vars".
Here you need to add the following vars:
* AWS_ACCESS_KEY_ID: The access key for your AWS account.
* AWS_SECRET_ACCESS_KEY: The secret key for your AWS account.
* AWS_STORAGE_BUCKET_NAME: Your Amazon Web Services storage bucket name.
* AWS_S3_REGION_NAME *Optional*: Name of the AWS S3 region to use (eg. eu-west-1)
* AWS_S3_ENDPOINT_URL *Optional*: Custom S3 URL to use when connecting to S3, including
scheme.
* AWS_S3_CUSTOM_DOMAIN *Optional*: Your custom domain where the files can be downloaded
from.
### Non AWS S3 providers
It is also possible to use non AWS, S3 providers like for example Digital Ocean. Below
you will find example settings if you want to connect to Digital Ocean Spaces.
* AWS_ACCESS_KEY_ID: The spaces API key.
* AWS_SECRET_ACCESS_KEY: The spaces API secret key.
* AWS_STORAGE_BUCKET_NAME: The name of your space.
* AWS_S3_REGION_NAME: Name of the Digital Ocean spaces region (eg. ams3)
* AWS_S3_ENDPOINT_URL: (eg. https://ams3.digitaloceanspaces.com)
* AWS_S3_CUSTOM_DOMAIN: (eg. name-of-your-space.ams3.digitaloceanspaces.com)
## Scaling
Even though the template runs on the free plan of Heroku by default, it can easily be
scaled up to fit your needs. We recommend scaling up if you are going to use Baserow
with more than one user simultaneously. You can scale up by changing the dyno type
and increase the amount of dyno's.
### Workers per dyno
To spare resources, every dyno has only one worker by default. If you are upgrading to
a standard 2x dyno, you can increase the amount of workers to 2. This can be done via
the Config Vars in the Settings. On the settings page, click on "Reveal Config Vars",
find the "BASEROW_AMOUNT_OF_WORKERS" var and set the value to 2.
You can roughly estimate the amount of workers based on the available RAM of your
dyno type. Every worker needs around 512MB ram, so a standard x1 dyno should have one
worker, a standard x2 can have 2 workers and a performance M can have 4.
### Postgres
By default, we are using the `heroku-postgresql:hobby-dev` addon because that supports
10k rows for free. If you need more rows, you need to upgrade that addon.
### Redis
By default, we are using the `heroku-redis:hobby` addon because that addon supports
20 connections for free. If you are scaling up, you need more connections which means
you need to upgrade that addon. In order to roughly estimate how many connections you
would need, you can do DYNO COUNT * BASEROW_AMOUNT_OF_WORKERS * 15.
## Custom Domain
If you added a custom domain, then you need to change a Config Var on the settings
page. Go to the Settings page and click on "Reveal Config Vars". Here you need to set
the `BASEROW_PUBLIC_URL` value and add your own URL. If your domain is
`baserow-test.com` ,then the value should be `https://baserow-test.com`. If you don't
have a custom domain then this value can be empty.
## Environment variables
* SECRET_KEY: A unique string that is used to generate secrets.
* BASEROW_PUBLIC_URL: The public URL of your Heroku Baserow app. If empty, the default
Heroku app URL is used, but if it differs it must be changed.
(eg. https://baserow-test.com).
* BASEROW_AMOUNT_OF_WORKERS: The amount of workers per dyno.
* AWS_ACCESS_KEY_ID: The spaces API key.
* AWS_SECRET_ACCESS_KEY: The spaces API secret key.
* AWS_STORAGE_BUCKET_NAME: The name of your space.
* AWS_S3_REGION_NAME: Name of the Digital Ocean spaces region (eg. ams3)
* AWS_S3_ENDPOINT_URL: Custom S3 URL to use when connecting to S3, including scheme.
* AWS_S3_CUSTOM_DOMAIN: Your custom domain where the files can be downloaded from.
## Update
When a new version of Baserow has been released, you probably want to update.
Unfortunately, it is not possible to update from a public repository via the web
interface, so you need to do that via the command line. If don't have the
heroku-cli installed, you can follow the instructions here
https://devcenter.heroku.com/articles/heroku-cli. After the cli is installed, you
probably need to run `heroku login` to authenticate with your account.
Make sure that you navigate to an empty directory and then run the commands below.
Don't forgot to replace `YOUR_APP_NAME` with the name of your Heroku app.
```
$ git clone --branch master https://gitlab.com/bramw/baserow.git
$ cd baserow
$ git remote add heroku https://git.heroku.com/YOUR_APP_NAME.git
$ git push -f heroku master
```
The latest version is deployed after the command finishes. You can optionally cleanup
the created directory by executing the following commands.
```
$ cd ../
$ rm -rf baserow
```

53
heroku.Dockerfile Normal file
View file

@ -0,0 +1,53 @@
FROM ubuntu:focal
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && \
apt install -y \
curl sudo gnupg2
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - \
&& curl -sL https://deb.nodesource.com/setup_12.x | bash -
RUN apt-get update && \
apt install -y \
make git nginx supervisor \
libpq-dev \
python3 build-essential libxslt-dev python3-dev python3-virtualenv \
python3-setuptools zlib1g-dev libffi-dev libssl-dev python3-pip \
nodejs \
&& rm -rf /var/cache/apt /var/lib/apt/lists
RUN npm install --global yarn mjml
RUN mkdir -p /baserow
WORKDIR /baserow
RUN service supervisor stop && service nginx stop
RUN rm -f /etc/nginx/sites-enabled/*
ADD . /baserow/baserow
RUN virtualenv -p python3 env
RUN env/bin/pip install --no-cache -r baserow/backend/requirements/base.txt
RUN env/bin/pip install dj-database-url boto3==1.16.25 django-storages==1.10.1
RUN (cd baserow/web-frontend && yarn install && yarn build)
RUN (mkdir -p /baserow/heroku/heroku && \
mkdir /baserow/media && \
touch /baserow/heroku/heroku/__init__.py)
ADD deploy/heroku/settings.py /baserow/heroku/heroku
ENV PYTHONPATH $PYTHONPATH:/baserow/baserow/backend/src:/baserow/heroku
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
ENV TMPDIR=/run/temp
ADD deploy/heroku/supervisor.conf /etc/supervisor/conf.d/supervisor.conf
RUN ln -sf /dev/stdout /var/log/supervisor/supervisord.log
ADD deploy/heroku/nginx.conf /baserow/nginx.conf
RUN ln -sf /dev/stdout /var/log/nginx/access.log
RUN ln -sf /dev/stderr /var/log/nginx/error.log
ADD deploy/heroku/entry.sh /baserow/entry.sh
RUN ["chmod", "+x", "/baserow/entry.sh"]

9
heroku.yml Normal file
View file

@ -0,0 +1,9 @@
build:
docker:
web: heroku.Dockerfile
release:
image: web
command:
- /baserow/env/bin/python /baserow/baserow/backend/src/baserow/manage.py migrate --settings=heroku.settings && /baserow/env/bin/python /baserow/baserow/backend/src/baserow/manage.py sync_templates --settings=heroku.settings
run:
web: . /baserow/entry.sh && /usr/bin/supervisord --configuration /etc/supervisor/conf.d/supervisor.conf