Resolve "Fix sandbox environment"

.gitlab-ci.yml

@@ -35,6 +35,7 @@ backend-flake8:
   script:
     - cd backend
     - make install-dependencies
+    - make install-dev-dependencies
     - make lint
 
 backend-pytest:
@@ -50,5 +51,6 @@ backend-pytest:
   script:
     - cd backend
     - make install-dependencies
+    - make install-dev-dependencies
     - export PYTHONPATH=$CI_PROJECT_DIR/backend/src
     - make test

README.md

@@ -1,75 +1,74 @@
 # Baserow
 
-To first start Baserow make sure you have installed docker and docker-compose. After that execute the following commands.
+Open source data collaboration platform.
+
+The complete toolchain for collaborating on any kind of data. One tool. Any device. Open
+Source. Extendible. And we’re just getting started.
+
+## Try out a demo
+
+If you just want to try out Baserow you can easily start a demo environment via 
+`docker-compose`. Just run the command following command and visit http://localhost:3000
+in your browser
+
+```
+$ docker network create baserow_demo_default
+$ docker-compose -f docker-compose.demo.yml up
+```
+
+## Plugin development
+
+If you want to start developing a plugin I recommend using the sandbox environment. In 
+there Baserow is installed as a dependency, exactly like and end user would have, if 
+your plugin works in the sandbox it will work for everyone. Execute the following 
+commands to get started. Note that the sandbox container might have a different name 
+like `baserow_sandbox_1`.
 
 ```
 $ docker network create baserow_default
 $ docker-compose up -d
+$ docker exec -it baserow_sandbox bash
+$ python src/sandbox/manage.py migrate
+$ python src/sandbox/manage.py runserver 0.0.0.0:8000
 ```
 
-If you just want to try out the application I recommend using the sandbox environment to start. First you need to open bash in the just created container and then you should navigate to the sandbox directory, start the django development server, start the nuxt development server and visit http://localhost:3000 in your browser. An example of the commands is described below.
+Open a new terminal window and execute the following commands.
 
 ```
-# terminal window 1
-$ docker exec -it baserow bash
-$ cd /baserow/sandbox
-$ yarn install
-$ yarn run dev
-
-# terminal window 2
-$ docker exec -it baserow bash
-$ cd /baserow/sandbox
-$ python manage.py runserver 0.0.0.0:8000
-```
-
-Both servers should be running and if you visit http://localhost:3000 you should see a working version of Baserow.
-
-## Development
-
-In order to start developing for the backend you need to execute the following commands.
-
-```
-docker exec -it baserow bash
-cd /baserow/backend/src/baserow
-python manage.py runserver 0.0.0.0:8000
-```
-
-In order to start developing for the web frontend you need to execute the following commands. Note that the backend server must also be running.
-
-```
-$ docker exec -it baserow bash
-$ cd /baserow/web-frontend
+$ docker exec -it baserow_sandbox bash
 $ yarn install
 $ yarn run dev
 ```
 
-When the development servers are on you can visit http://localhost:3000.
+Now you'll have Baserow running, but installed as a dependency. To avoid conflicts the
+backend now runs on port 8001 and the web-frontend on 3001, so if you visit 
+http://localhost:3001 in your browser you should see a working environment. You can 
+change the settings to add a Django app or Nuxt module. More documentation and a 
+boilerplate for plugin development are going to follow soon.
 
-## Testing and linting
+## Core development
 
-There are a few commands you can use inside the container to test and lint parts of the code.
+If you want to setup the development environment for core Baserow development you have 
+to execute the following commands to start the backend part. Note that the sandbox 
+container might have a different name like `backend_1`.
 
 ```
-$ docker exec -it baserow bash
-$ cd /baserow
-
-# run pytest for the backend
-$ cd backend
-$ make lint
-
-# run flake8 for the backend
-$ cd backend
-$ make test
-
-# run jest for the web frontend
-$ cd web-frontend
-$ make test
-
-# run eslint for the web frontend
-$ cd web-frontend
-$ make eslint
-
-# run stylelint for the web frontend
-$ cd web-frontend
-$ make stylelint
+$ docker network create baserow_default
+$ docker-compose up -d
+$ docker exec -it backend bash
+$ python src/baserow/manage.py migrate
+$ python src/baserow/manage.py runserver 0.0.0.0:8000
 ```
+
+In order to start the web-frontend environment you may execute the following commands. 
+Note that the sandbox container might have a different name like `web-frontend_1`.
+
+```
+$ docker network create baserow_default
+$ docker exec -it web-frontend bash
+$ yarn install
+$ yarn dev
+```
+
+Now you'll have the Baserow development environment running. Visit http://localhost:3000
+in your browser and you should see a working version in development mode.

backend/Dockerfile

@@ -1,12 +0,0 @@
-FROM python:3.6
-
-ADD . /backend
-
-WORKDIR /backend
-
-ENV PYTHONPATH $PYTHONPATH:/backend/src
-
-RUN apt-get update && apt-get -y install make curl gnupg2
-RUN make install-dependencies
-
-CMD tail -f /dev/null

backend/Dockerfile.demo

@@ -0,0 +1,17 @@
+FROM python:3.6
+
+ADD . /backend
+
+WORKDIR /backend
+
+ENV PYTHONPATH $PYTHONPATH:/backend/src
+ENV DJANGO_SETTINGS_MODULE='baserow.config.settings.demo'
+
+RUN apt-get update
+RUN apt-get -y install make
+RUN apt-get -y install curl
+RUN apt-get -y install gnupg2
+
+RUN make install-dependencies
+
+ENTRYPOINT python src/baserow/manage.py migrate && python src/baserow/manage.py runserver 0.0.0.0:8000

backend/Dockerfile.dev

@@ -0,0 +1,18 @@
+FROM python:3.6
+
+ADD . /backend
+
+WORKDIR /backend
+
+ENV PYTHONPATH $PYTHONPATH:/backend/src
+ENV DJANGO_SETTINGS_MODULE='baserow.config.settings.dev'
+
+RUN apt-get update
+RUN apt-get -y install make
+RUN apt-get -y install curl
+RUN apt-get -y install gnupg2
+
+RUN make install-dependencies
+RUN make install-dev-dependencies
+
+CMD tail -f /dev/null

backend/Makefile

@@ -1,6 +1,8 @@
 install-dependencies:
 	apt-get update && apt-get install -y libpq-dev postgresql postgresql-contrib
 	pip install -r requirements/base.txt
+
+install-dev-dependencies:
 	pip install -r requirements/dev.txt
 
 lint:

backend/src/baserow/config/settings/base.py

@@ -10,7 +10,7 @@ SECRET_KEY = '0s0@+(p7&qip7spd%l1$sp%ne*e(2hk1jv!6&mq^d+&40m+24j'
 # SECURITY WARNING: don't run with debug turned on in production!
 DEBUG = False
 
-ALLOWED_HOSTS = ['localhost', 'backend']
+ALLOWED_HOSTS = ['localhost', 'backend', 'sandbox']
 
 
 INSTALLED_APPS = [
@@ -128,7 +128,9 @@ REST_FRAMEWORK = {
 
 CORS_ORIGIN_WHITELIST = (
     'http://localhost:3000',
-    'http://backend:3000'
+    'http://localhost:3001',
+    'http://backend:8000',
+    'http://sandbox:8000'
 )
 
 JWT_AUTH = {

backend/src/baserow/config/settings/demo.py

@@ -0,0 +1,22 @@
+import datetime
+from .base import *  # noqa: F403, F401
+
+
+JWT_AUTH = {
+    'JWT_EXPIRATION_DELTA': datetime.timedelta(days=1),
+    'JWT_ALLOW_REFRESH': True,
+    'JWT_REFRESH_EXPIRATION_DELTA': datetime.timedelta(days=7),
+    'JWT_RESPONSE_PAYLOAD_HANDLER': 'baserow.api.v0.user.jwt.'
+                                    'jwt_response_payload_handler'
+}
+
+DATABASES = {
+    'default': {
+        'ENGINE': 'django.db.backends.postgresql_psycopg2',
+        'NAME': 'baserow',
+        'USER': 'baserow',
+        'PASSWORD': 'baserow',
+        'HOST': 'db',
+        'PORT': '5432',
+    }
+}

backend/src/baserow/config/settings/dev.py

@@ -1,5 +1,6 @@
 from .base import *  # noqa: F403, F401
 
+
 DEBUG = True
 
 try:

backend/src/baserow/contrib/database/fields/field_types.py

@@ -72,8 +72,8 @@ class NumberFieldType(FieldType):
     def random_value(self, instance, fake):
         if instance.number_type == NUMBER_TYPE_INTEGER:
             return fake.pyint(
-                min=-10000 if instance.number_negative else 0,
-                max=10000,
+                min_value=-10000 if instance.number_negative else 0,
+                max_value=10000,
                 step=1
             )
         elif instance.number_type == NUMBER_TYPE_DECIMAL:

backend/src/baserow/manage.py

@@ -1,11 +1,8 @@
 #!/usr/bin/env python
-"""Django's command-line utility for administrative tasks."""
-import os
 import sys
 
 
 def main():
-    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'baserow.config.settings.dev')
     try:
         from django.core.management import execute_from_command_line
     except ImportError as exc:

docker-compose.demo.yml

@@ -0,0 +1,36 @@
+version: "3"
+
+services:
+  db:
+    image: postgres:11.3
+    environment:
+      - POSTGRES_USER=baserow
+      - POSTGRES_PASSWORD=baserow
+      - POSTGRES_DB=baserow
+    ports:
+      - 5432:5432
+
+  backend:
+    container_name: backend
+    build:
+      context: ./backend/
+      dockerfile: Dockerfile.demo
+    ports:
+      - 8000:8000
+    depends_on:
+      - db
+
+  web-frontend:
+    container_name: web-frontend
+    build:
+      context: ./web-frontend/
+      dockerfile: Dockerfile.demo
+    ports:
+      - 3000:3000
+    depends_on:
+      - backend
+
+networks:
+  default:
+    external:
+      name: baserow_demo_default

docker-compose.yml

@@ -12,7 +12,9 @@ services:
 
   backend:
     container_name: backend
-    build: ./backend/.
+    build:
+      context: ./backend/
+      dockerfile: Dockerfile.dev
     volumes:
       - ./backend:/backend
     ports:
@@ -22,7 +24,9 @@ services:
 
   web-frontend:
     container_name: web-frontend
-    build: ./web-frontend/.
+    build:
+      context: ./web-frontend/
+      dockerfile: Dockerfile.dev
     volumes:
       - ./web-frontend:/web-frontend
     ports:
@@ -30,7 +34,14 @@ services:
     depends_on:
       - backend
 
-networks:
-  default:
-    external:
-      name: baserow_default
+  sandbox:
+    build:
+      context: ./
+      dockerfile: ./sandbox/Dockerfile
+    volumes:
+      - ./web-frontend:/web-frontend
+      - ./backend:/backend
+      - ./sandbox:/sandbox
+    ports:
+      - 8001:8000
+      - 3001:3000

sandbox/Dockerfile

@@ -0,0 +1,24 @@
+FROM python:3.6
+
+RUN apt-get update
+RUN apt-get -y install make
+RUN apt-get -y install curl
+RUN apt-get -y install gnupg2
+
+RUN curl -sL https://deb.nodesource.com/setup_10.x  | bash -
+RUN apt-get -y install nodejs
+RUN npm install -g yarn
+
+ADD ./backend /backend
+ADD ./web-frontend /web-frontend
+ADD ./sandbox /sandbox
+
+WORKDIR /backend
+RUN make install-dependencies
+RUN make install-dev-dependencies
+
+ENV PYTHONPATH $PYTHONPATH:/backend/src:/sandbox/src
+ENV DJANGO_SETTINGS_MODULE='sandbox.settings'
+
+WORKDIR /sandbox
+CMD tail -f /dev/null

sandbox/nuxt.config.js

@@ -1,5 +1,11 @@
-import path from 'path'
+import _ from 'lodash'
+import base from 'baserow/config/nuxt.config.base.js'
 
-import production from '../web-frontend/config/nuxt.config.production.js'
+const config = {
+  env: {
+    baseUrl: 'http://sandbox:8000/api/v0',
+    publicBaseUrl: 'http://localhost:8001/api/v0',
+  },
+}
 
-export default production(path.resolve(__dirname))
+export default _.assign(base('baserow'), config)

sandbox/package.json

@@ -4,10 +4,9 @@
   "scripts": {
     "build": "nuxt build",
     "dev": "nuxt --hostname 0.0.0.0",
-    "generate": "nuxt generate",
-    "start": "nuxt start"
+    "start": "nuxt start --hostname 0.0.0.0"
   },
   "dependencies": {
-    "baserow": "link:../web-frontend"
+    "baserow": "file:/web-frontend"
   }
 }

sandbox/src/sandbox/settings.py

@@ -0,0 +1,4 @@
+from baserow.config.settings.base import *  # noqa: F403, F401
+
+
+DEBUG = True

web-frontend/Dockerfile

@@ -1,10 +0,0 @@
-FROM node:10
-
-ADD . /web-frontend
-
-WORKDIR /web-frontend
-
-RUN apt-get update && apt-get -y install make curl gnupg2
-RUN make install-dependencies
-
-CMD tail -f /dev/null

web-frontend/Dockerfile.demo

@@ -0,0 +1,16 @@
+FROM node:10
+
+ADD . /web-frontend
+
+WORKDIR /web-frontend
+
+RUN apt-get update
+RUN apt-get -y install make
+RUN apt-get -y install curl
+RUN apt-get -y install gnupg2
+
+RUN make install-dependencies
+RUN yarn install
+RUN yarn run build
+
+ENTRYPOINT yarn run demo

web-frontend/Dockerfile.dev

@@ -0,0 +1,12 @@
+FROM node:10
+
+ADD . /web-frontend
+
+WORKDIR /web-frontend
+
+RUN apt-get update
+RUN apt-get -y install make
+RUN apt-get -y install curl
+RUN apt-get -y install gnupg2
+
+CMD tail -f /dev/null

web-frontend/config/nuxt.config.base.js

@@ -1,12 +1,17 @@
-export default {
-  modules: ['@/modules/core/module.js', '@/modules/database/module.js'],
+export default function (base = '@') {
+  return {
+    modules: [
+      base + '/modules/core/module.js',
+      base + '/modules/database/module.js',
+    ],
 
-  env: {
-    // The API base url, this will be prepended to the urls of the remote calls.
-    baseUrl: 'http://backend:8000/api/v0',
+    env: {
+      // The API base url, this will be prepended to the urls of the remote calls.
+      baseUrl: 'http://backend:8000/api/v0',
 
-    // If the API base url must different at the client side it can be changed
-    // here.
-    publicBaseUrl: 'http://localhost:8000/api/v0',
-  },
+      // If the API base url must different at the client side it can be changed
+      // here.
+      publicBaseUrl: 'http://localhost:8000/api/v0',
+    },
+  }
 }

web-frontend/config/nuxt.config.demo.js

@@ -0,0 +1,3 @@
+import base from './nuxt.config.base.js'
+
+export default base()

web-frontend/config/nuxt.config.dev.js

@@ -3,7 +3,7 @@ import StyleLintPlugin from 'stylelint-webpack-plugin'
 
 import base from './nuxt.config.base.js'
 
-const config = {
+export default _.assign(base(), {
   build: {
     extend(config, ctx) {
       if (ctx.isDev && ctx.isClient) {
@@ -22,6 +22,4 @@ const config = {
       }),
     ],
   },
-}
-
-export default _.assign(base, config)
+})

web-frontend/config/nuxt.config.production.js

@@ -1,20 +0,0 @@
-import path from 'path'
-
-import base from './nuxt.config.base.js'
-
-export default function (rootDir) {
-  const _ = require(rootDir + '/node_modules/lodash')
-
-  /**
-   * Because the nuxt source files are located in the web-frontend directory,
-   * but the project is started from another directory we have to explicitly set
-   * the source directory which contains the nuxt files and the root directory
-   * which contains the node modules.
-   */
-  const config = {
-    rootDir,
-    srcDir: path.resolve(__dirname, '../'),
-  }
-
-  return _.assign(base, config)
-}

web-frontend/config/nuxt.config.test.js

@@ -3,7 +3,7 @@ import _ from 'lodash'
 
 import base from './nuxt.config.base.js'
 
-const config = {
+export default _.assign({}, base(), {
   rootDir: resolve(__dirname, '../'),
   css: [],
   dev: false,
@@ -12,6 +12,4 @@ const config = {
     // The API base url, this will be prepended to the urls of the remote calls.
     baseUrl: 'http://localhost/api/v0',
   },
-}
-
-export default _.assign({}, base, config)
+})

web-frontend/modules/core/middleware/groupsAndApplications.js

@@ -4,7 +4,7 @@ import { getGroupCookie } from '@baserow/modules/core/utils/group'
  * This middleware will make sure that all the groups and applications belonging to
  * the user are fetched and added to the store.
  */
-export default async function ({ store, req, app }) {
+export default async function GroupsAndApplications({ store, req, app }) {
   // If nuxt generate, pass this middleware
   if (process.server && !req) return
 
@@ -19,7 +19,9 @@ export default async function ({ store, req, app }) {
 
       // If there is a groupId cookie we will select that group.
       if (groupId) {
-        await store.dispatch('group/selectById', groupId)
+        try {
+          await store.dispatch('group/selectById', groupId)
+        } catch {}
       }
     }
     // If the applications haven't been loaded we will also load them all.

web-frontend/package.json

@@ -1,14 +1,14 @@
 {
   "name": "baserow",
-  "version": "1.0.0",
+  "version": "0.0.1",
   "private": true,
   "description": "Baserow web frontend",
   "author": "Bram Wiepjes (Cloud 3)",
   "scripts": {
     "build": "nuxt build",
     "dev": "nuxt --hostname 0.0.0.0",
-    "generate": "nuxt generate",
     "start": "nuxt start --hostname 0.0.0.0",
+    "demo": "nuxt start --hostname 0.0.0.0 --config-file config/nuxt.config.demo.js",
     "eslint": "eslint --ext .js,.vue .",
     "stylelint": "stylelint **/*.scss --syntax scss",
     "jest": "jest -i --verbose false test/"