docker-compose up airflow-init produces airflow command error · Issue #14365 · apache/airflow

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

有情有义的沙滩裤 · Error when trying to ...· 1小时前 ·

果断的毛豆 · Python3返回意外的Unicode/ASCII· 8 月前 ·

空虚的花卷 · android ...· 11 月前 ·

骑白马的大熊猫 · redislive 时间不对 redis ...· 12 月前 ·

烦恼的上铺 · c# filestream delete ...· 1 年前 ·

体贴的风衣 · WPF入門教學系列二十七 ...· 1 年前 ·

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OS (e.g. from /etc/os-release): macOS 10.15.7

Kernel (e.g. uname -a): Darwin Kernel Version 19.6.0

Install tools: docker-compose

What happened :
Following the steps in the Quick start > Running Airflow in Docker page ( https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#initializing-environment ):

Pulled down the pre-made docker-compose.yaml file ( https://airflow.apache.org/docs/apache-airflow/2.0.1/docker-compose.yaml )

Ran


    docker-compose up airflow-init

within the same directory as the docker-compose file

This command failed with: airflow command error: argument GROUP_OR_COMMAND:


    airflow upgradedb

command, has been removed, please use


    airflow db upgrade

, see help above.

What you expected to happen :
The latest stable official airflow docker image to be pulled, and airflow-init to complete successfully.

How to reproduce it :
For the exact traceback block below, I stopped and removed all of my docker images:


    docker stop (docker ps -q)


    docker rm (docker ps -a -q)


    docker images -a -q | xargs docker rmi -f

Follow the instructions per the quick start docs ( https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html ) to pull down the docker-compose.yaml file, then attempt to run


    docker-compose up airflow-init

Anything else we need to know :
This is very similar issue to #14143

Below is the traceback block

~/C/P/docker-apache-airflow  docker-compose up airflow-init c̶o̶n̶d̶a̶-̶e̶n̶v̶
Pulling postgres (postgres:13)...
13: Pulling from library/postgres
45b42c59be33: Already exists
40adec129f1a: Pull complete
b4c431d00c78: Pull complete
2696974e2815: Pull complete
564b77596399: Pull complete
5044045cf6f2: Pull complete
d736e67e6ac3: Pull complete
390c1c9a5ae4: Pull complete
c0e62f172284: Pull complete
ebcdc659c5bf: Pull complete
29be22cb3acc: Pull complete
f63c47038e66: Pull complete
77a0c198cde5: Pull complete
c8752d5b785c: Pull complete
Digest: sha256:5cfcfb0516ddd317109e2746a462b1dc398d152936640a2b69284629f48dc288
Status: Downloaded newer image for postgres:13
Pulling redis (redis:latest)...
latest: Pulling from library/redis
45b42c59be33: Already exists
5ce2e937bf62: Pull complete
2a031498ff58: Pull complete
78f77a50d3fe: Pull complete
60553ab7cb08: Pull complete
68240431d2fc: Pull complete
Digest: sha256:f29bcfb891678a0c6a0fc5da0b32ce1ac685af87c0f3aa9327e562da8d3f3b88
Status: Downloaded newer image for redis:latest
Pulling airflow-init (apache/airflow:2.0.1)...
2.0.1: Pulling from apache/airflow
45b42c59be33: Already exists
f875e16ab19c: Already exists
a4fb13de3215: Pull complete
f3dd9aaf319e: Pull complete
451a8f7ee895: Pull complete
e2d0adf49357: Pull complete
52856765f525: Pull complete
fd428150aaa6: Pull complete
b4fa7f6b9775: Pull complete
b34f0a9234cb: Pull complete
537882d8c1ac: Pull complete
dffdc965d748: Pull complete
a97e5a67cb46: Pull complete
c401f5284516: Pull complete
af49f5db1642: Pull complete
c61449868181: Pull complete
0b574505c97e: Pull complete
a65349ef32b2: Pull complete
Digest: sha256:2bf4fe559e9e699487a50d4e587358bb0b3c90b59c8fefeccc8a4fb2e83f3d09
Status: Downloaded newer image for apache/airflow:2.0.1
Creating docker-apache-airflow_postgres_1 ... done
Creating docker-apache-airflow_redis_1 ... done
Creating docker-apache-airflow_airflow-init_1 ... done
Attaching to docker-apache-airflow_airflow-init_1
airflow-init_1 | BACKEND=postgresql+psycopg2
airflow-init_1 | DB_HOST=postgres
airflow-init_1 | DB_PORT=5432
airflow-init_1 |
airflow-init_1 | DB: postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init_1 | [2021-02-22 18:19:41,935] {db.py:674} INFO - Creating tables
airflow-init_1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
airflow-init_1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
airflow-init_1 | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors
airflow-init_1 | yield
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 356, in upgrade_revs
airflow-init_1 | revs = list(revs)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 904, in iterate_revisions
airflow-init_1 | requested_lowers = self.get_revisions(lower)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 455, in get_revisions
airflow-init_1 | return sum([self.get_revisions(id_elem) for id_elem in id ], ())
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 455, in
airflow-init_1 | return sum([self.get_revisions(id_elem) for id_elem in id ], ())
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 460, in get_revisions
airflow-init_1 | for rev_id in resolved_id
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 460, in
airflow-init_1 | for rev_id in resolved_id
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 536, in _revision_for_ident
airflow-init_1 | resolved_id,
airflow-init_1 | alembic.script.revision.ResolutionError: No such revision or branch '449b4072c2da'
airflow-init_1 |
airflow-init_1 | The above exception was the direct cause of the following exception:
airflow-init_1 |
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/bin/airflow", line 8, in
airflow-init_1 | sys.exit(main())
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/ main .py", line 40, in main
airflow-init_1 | args.func(args)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, in command
airflow-init_1 | return func(*args, **kwargs)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/cli.py", line 89, in wrapper
airflow-init_1 | return f(*args, **kwargs)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/commands/db_command.py", line 48, in upgradedb
airflow-init_1 | db.upgradedb()
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 684, in upgradedb
airflow-init_1 | command.upgrade(config, 'heads')
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/command.py", line 294, in upgrade
airflow-init_1 | script.run_env()
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 481, in run_env
airflow-init_1 | util.load_python_file(self.dir, "env.py")
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
airflow-init_1 | module = load_module_py(module_id, path)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 182, in load_module_py
airflow-init_1 | spec.loader.exec_module(module)
airflow-init_1 | File "", line 678, in exec_module
airflow-init_1 | File "", line 219, in _call_with_frames_removed
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 108, in
airflow-init_1 | run_migrations_online()
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
airflow-init_1 | context.run_migrations()
airflow-init_1 | File "", line 8, in run_migrations
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
airflow-init_1 | self.get_context().run_migrations(**kw)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 548, in run_migrations
airflow-init_1 | for step in self._migrations_fn(heads, self):
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/command.py", line 283, in upgrade
airflow-init_1 | return script. upgrade_revs(revision, rev)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 361, in upgrade_revs
airflow-init_1 | for script in reversed(list(revs))
airflow-init_1 | File "/usr/local/lib/python3.6/contextlib.py", line 99, in exit
airflow-init_1 | self.gen.throw(type, value, traceback)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 194, in catch_revision_errors
airflow-init_1 | compat.raise (util.CommandError(resolution), from =re)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 294, in raise
airflow-init_1 | raise exception
airflow-init_1 | alembic.util.exc.CommandError: Can't locate revision identified by '449b4072c2da'
airflow-init_1 | usage: airflow [-h] GROUP_OR_COMMAND ...
airflow-init_1 |
airflow-init_1 | positional arguments:
airflow-init_1 | GROUP_OR_COMMAND
airflow-init_1 |
airflow-init_1 | Groups:
airflow-init_1 | celery Celery components
airflow-init_1 | config View configuration
airflow-init_1 | connections Manage connections
airflow-init_1 | dags Manage DAGs
airflow-init_1 | db Database operations
airflow-init_1 | kubernetes Tools to help run the KubernetesExecutor
airflow-init_1 | pools Manage pools
airflow-init_1 | providers Display providers
airflow-init_1 | roles Manage roles
airflow-init_1 | tasks Manage tasks
airflow-init_1 | users Manage users
airflow-init_1 | variables Manage variables
airflow-init_1 |
airflow-init_1 | Commands:
airflow-init_1 | cheat-sheet Display cheat sheet
airflow-init_1 | info Show information about current Airflow and environment
airflow-init_1 | kerberos Start a kerberos ticket renewer
airflow-init_1 | plugins Dump information about loaded plugins
airflow-init_1 | rotate-fernet-key
airflow-init_1 | Rotate encrypted connection credentials and variables
airflow-init_1 | scheduler Start a scheduler instance
airflow-init_1 | sync-perm Update permissions for existing roles and DAGs
airflow-init_1 | version Show the version
airflow-init_1 | webserver Start a Airflow webserver instance
airflow-init_1 |
airflow-init_1 | optional arguments:
airflow-init_1 | -h, --help show this help message and exit
airflow-init_1 |
airflow-init_1 | airflow command error: argument GROUP_OR_COMMAND: airflow upgradedb command, has been removed, please use airflow db upgrade , see help above.
airflow-init_1 | [2021-02-22 18:19:50,692] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-22 18:19:51,231] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-22 18:19:52,417] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-22 18:19:52,514] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | airflow already exist in the db
airflow-init_1 | 2.0.1
docker-apache-airflow_airflow-init_1 exited with code 0

@chrisrfu It's hard to tell exactly what's going wrong here, but it appears that there's some type of mismatch between old/new data or old/new versions.

For example, alembic.script.revision.ResolutionError: No such revision or branch '449b4072c2da' seems to refer to an existing alembic revision number. It could also be that you need to upgrade docker.

How are you using conda-env ? Is that an existing environment file?

Looks like you have tried installing different version of Airflow, maybe you tried to install the image for Airflow Master and then rolled back to 2.0.1

docker-compose down --volumes --rmi all
and then re-run your original commands.
I can tell you tried with Master because it is complaining about alembic version: 449b4072c2da  -- which is only available  in Master.
          i ran docker-compose down --volumes --rmi all and then docker images -a -q | xargs docker rmi -f

for good measure and re-ran my original commands and still no dice
Pulling postgres (postgres:13)...

13: Pulling from library/postgres

45b42c59be33: Already exists

40adec129f1a: Pull complete

b4c431d00c78: Pull complete

2696974e2815: Pull complete

564b77596399: Pull complete

5044045cf6f2: Pull complete

d736e67e6ac3: Pull complete

390c1c9a5ae4: Pull complete

c0e62f172284: Pull complete

ebcdc659c5bf: Pull complete

29be22cb3acc: Pull complete

f63c47038e66: Pull complete

77a0c198cde5: Pull complete

c8752d5b785c: Pull complete

Digest: sha256:5cfcfb0516ddd317109e2746a462b1dc398d152936640a2b69284629f48dc288

Status: Downloaded newer image for postgres:13

Pulling redis (redis:latest)...

latest: Pulling from library/redis

45b42c59be33: Already exists

5ce2e937bf62: Pull complete

2a031498ff58: Pull complete

7407cbab5258: Pull complete

c2d6c3b35be8: Pull complete

1997ce073dcd: Pull complete

Digest: sha256:26bd5f3d2bbfb21d0e4f03d642570e63c8776d2a9e5963e4057280fb3b4e3e29

Status: Downloaded newer image for redis:latest

Pulling airflow-init (apache/airflow:2.0.1)...

2.0.1: Pulling from apache/airflow

45b42c59be33: Already exists

f875e16ab19c: Already exists

a4fb13de3215: Pull complete

f3dd9aaf319e: Pull complete

451a8f7ee895: Pull complete

e2d0adf49357: Pull complete

52856765f525: Pull complete

fd428150aaa6: Pull complete

b4fa7f6b9775: Pull complete

b34f0a9234cb: Pull complete

537882d8c1ac: Pull complete

dffdc965d748: Pull complete

a97e5a67cb46: Pull complete

c401f5284516: Pull complete

af49f5db1642: Pull complete

c61449868181: Pull complete

0b574505c97e: Pull complete

a65349ef32b2: Pull complete

Digest: sha256:2bf4fe559e9e699487a50d4e587358bb0b3c90b59c8fefeccc8a4fb2e83f3d09

Status: Downloaded newer image for apache/airflow:2.0.1

Creating docker-apache-airflow_postgres_1 ... done

Creating docker-apache-airflow_redis_1    ... done

Creating docker-apache-airflow_airflow-init_1 ... done

Attaching to docker-apache-airflow_airflow-init_1

airflow-init_1       | BACKEND=postgresql+psycopg2

airflow-init_1       | DB_HOST=postgres

airflow-init_1       | DB_PORT=5432

airflow-init_1       |

airflow-init_1       | DB: postgresql+psycopg2://airflow:***@postgres/airflow

airflow-init_1       | [2021-02-24 15:11:24,568] {db.py:674} INFO - Creating tables

airflow-init_1       | INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.

airflow-init_1       | INFO  [alembic.runtime.migration] Will assume transactional DDL.

airflow-init_1       | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | Traceback (most recent call last):

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors

airflow-init_1       |     yield

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 356, in upgrade_revs

airflow-init_1       |     revs = list(revs)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 904, in iterate_revisions

airflow-init_1       |     requested_lowers = self.get_revisions(lower)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 455, in get_revisions

airflow-init_1       |     return sum([self.get_revisions(id_elem) for id_elem in id], ())

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 455, in 

airflow-init_1       |     return sum([self.get_revisions(id_elem) for id_elem in id], ())

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 460, in get_revisions

airflow-init_1       |     for rev_id in resolved_id

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 460, in 

airflow-init_1       |     for rev_id in resolved_id

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 536, in _revision_for_ident

airflow-init_1       |     resolved_id,

airflow-init_1       | alembic.script.revision.ResolutionError: No such revision or branch '449b4072c2da'

airflow-init_1       |

airflow-init_1       | The above exception was the direct cause of the following exception:

airflow-init_1       |

airflow-init_1       | Traceback (most recent call last):

airflow-init_1       |   File "/home/airflow/.local/bin/airflow", line 8, in 

airflow-init_1       |     sys.exit(main())

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/airflow/main.py", line 40, in main

airflow-init_1       |     args.func(args)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, in command

airflow-init_1       |     return func(*args, **kwargs)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/cli.py", line 89, in wrapper

airflow-init_1       |     return f(*args, **kwargs)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/commands/db_command.py", line 48, in upgradedb

airflow-init_1       |     db.upgradedb()

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 684, in upgradedb

airflow-init_1       |     command.upgrade(config, 'heads')

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/command.py", line 294, in upgrade

airflow-init_1       |     script.run_env()

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 481, in run_env

airflow-init_1       |     util.load_python_file(self.dir, "env.py")

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file

airflow-init_1       |     module = load_module_py(module_id, path)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 182, in load_module_py

airflow-init_1       |     spec.loader.exec_module(module)

airflow-init_1       |   File "", line 678, in exec_module

airflow-init_1       |   File "", line 219, in _call_with_frames_removed

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 108, in 

airflow-init_1       |     run_migrations_online()

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online

airflow-init_1       |     context.run_migrations()

airflow-init_1       |   File "", line 8, in run_migrations

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 813, in run_migrations

airflow-init_1       |     self.get_context().run_migrations(**kw)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 548, in run_migrations

airflow-init_1       |     for step in self._migrations_fn(heads, self):

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/command.py", line 283, in upgrade

airflow-init_1       |     return script.upgrade_revs(revision, rev)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 361, in upgrade_revs

airflow-init_1       |     for script in reversed(list(revs))

airflow-init_1       |   File "/usr/local/lib/python3.6/contextlib.py", line 99, in exit

airflow-init_1       |     self.gen.throw(type, value, traceback)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 194, in catch_revision_errors

airflow-init_1       |     compat.raise(util.CommandError(resolution), from=re)

airflow-init_1       |   File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 294, in raise

airflow-init_1       |     raise exception

airflow-init_1       | alembic.util.exc.CommandError: Can't locate revision identified by '449b4072c2da'

airflow-init_1       | usage: airflow [-h] GROUP_OR_COMMAND ...

airflow-init_1       |

airflow-init_1       | positional arguments:

airflow-init_1       |   GROUP_OR_COMMAND

airflow-init_1       |

airflow-init_1       |     Groups:

airflow-init_1       |       celery         Celery components

airflow-init_1       |       config         View configuration

airflow-init_1       |       connections    Manage connections

airflow-init_1       |       dags           Manage DAGs

airflow-init_1       |       db             Database operations

airflow-init_1       |       kubernetes     Tools to help run the KubernetesExecutor

airflow-init_1       |       pools          Manage pools

airflow-init_1       |       providers      Display providers

airflow-init_1       |       roles          Manage roles

airflow-init_1       |       tasks          Manage tasks

airflow-init_1       |       users          Manage users

airflow-init_1       |       variables      Manage variables

airflow-init_1       |

airflow-init_1       |     Commands:

airflow-init_1       |       cheat-sheet    Display cheat sheet

airflow-init_1       |       info           Show information about current Airflow and environment

airflow-init_1       |       kerberos       Start a kerberos ticket renewer

airflow-init_1       |       plugins        Dump information about loaded plugins

airflow-init_1       |       rotate-fernet-key

airflow-init_1       |                      Rotate encrypted connection credentials and variables

airflow-init_1       |       scheduler      Start a scheduler instance

airflow-init_1       |       sync-perm      Update permissions for existing roles and DAGs

airflow-init_1       |       version        Show the version

airflow-init_1       |       webserver      Start a Airflow webserver instance

airflow-init_1       |

airflow-init_1       | optional arguments:

airflow-init_1       |   -h, --help         show this help message and exit

airflow-init_1       |

airflow-init_1       | airflow command error: argument GROUP_OR_COMMAND: airflow upgradedb command, has been removed, please use airflow db upgrade, see help above.

airflow-init_1       | [2021-02-24 15:11:31,808] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | [2021-02-24 15:11:32,232] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | [2021-02-24 15:11:33,231] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | [2021-02-24 15:11:33,303] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | airflow already exist in the db

airflow-init_1       | 2.0.1

docker-apache-airflow_airflow-init_1 exited with code 0
which was solved by:
instead of docker stop (docker ps -q), docker rm (docker ps -a -q), and then docker-compose down --volumes --rm all

just going with docker-compose down --volumes --rm alland then re-running my original commands
Creating network "docker-apache-airflow-201_default" with the default driver

Creating volume "docker-apache-airflow-201_postgres-db-volume" with default driver

Pulling postgres (postgres:13)...

13: Pulling from library/postgres

45b42c59be33: Already exists

40adec129f1a: Pull complete

b4c431d00c78: Pull complete

2696974e2815: Pull complete

564b77596399: Pull complete

5044045cf6f2: Pull complete

d736e67e6ac3: Pull complete

390c1c9a5ae4: Pull complete

c0e62f172284: Pull complete

ebcdc659c5bf: Pull complete

29be22cb3acc: Pull complete

f63c47038e66: Pull complete

77a0c198cde5: Pull complete

c8752d5b785c: Pull complete

Digest: sha256:5cfcfb0516ddd317109e2746a462b1dc398d152936640a2b69284629f48dc288

Status: Downloaded newer image for postgres:13

Pulling redis (redis:latest)...

latest: Pulling from library/redis

45b42c59be33: Already exists

5ce2e937bf62: Pull complete

2a031498ff58: Pull complete

7407cbab5258: Pull complete

c2d6c3b35be8: Pull complete

1997ce073dcd: Pull complete

Digest: sha256:26bd5f3d2bbfb21d0e4f03d642570e63c8776d2a9e5963e4057280fb3b4e3e29

Status: Downloaded newer image for redis:latest

Pulling airflow-init (apache/airflow:2.0.1)...

2.0.1: Pulling from apache/airflow

45b42c59be33: Already exists

f875e16ab19c: Already exists

a4fb13de3215: Pull complete

f3dd9aaf319e: Pull complete

451a8f7ee895: Pull complete

e2d0adf49357: Pull complete

52856765f525: Pull complete

fd428150aaa6: Pull complete

b4fa7f6b9775: Pull complete

b34f0a9234cb: Pull complete

537882d8c1ac: Pull complete

dffdc965d748: Pull complete

a97e5a67cb46: Pull complete

c401f5284516: Pull complete

af49f5db1642: Pull complete

c61449868181: Pull complete

0b574505c97e: Pull complete

a65349ef32b2: Pull complete

Digest: sha256:2bf4fe559e9e699487a50d4e587358bb0b3c90b59c8fefeccc8a4fb2e83f3d09

Status: Downloaded newer image for apache/airflow:2.0.1

Creating docker-apache-airflow-201_postgres_1 ... done

Creating docker-apache-airflow-201_redis_1    ... done

Creating docker-apache-airflow-201_airflow-init_1 ... done

Attaching to docker-apache-airflow-201_airflow-init_1

airflow-init_1       | BACKEND=postgresql+psycopg2

airflow-init_1       | DB_HOST=postgres

airflow-init_1       | DB_PORT=5432

airflow-init_1       |

airflow-init_1       | [2021-02-24 16:23:54,595] {cli_action_loggers.py:105} WARNING - Failed to log action with (psycopg2.errors.UndefinedTable) relation "log" does not exist

airflow-init_1       | LINE 1: INSERT INTO log (dttm, dag_id, task_id, event, execution_dat...

airflow-init_1       |                     ^

airflow-init_1       |

airflow-init_1       | [SQL: INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (%(dttm)s, %(dag_id)s, %(task_id)s, %(event)s, %(execution_date)s, %(owner)s, %(extra)s) RETURNING log.id]

airflow-init_1       | [parameters: {'dttm': datetime.datetime(2021, 2, 24, 16, 23, 54, 583379, tzinfo=Timezone('UTC')), 'dag_id': None, 'task_id': None, 'event': 'cli_upgradedb', 'execution_date': None, 'owner': 'airflow', 'extra': '{"host_name": "93ea84d65b76", "full_command": "['/home/airflow/.local/bin/airflow', 'db', 'upgrade']"}'}]

airflow-init_1       | (Background on this error at: http://sqlalche.me/e/13/f405)

airflow-init_1       | DB: postgresql+psycopg2://airflow:***@postgres/airflow

airflow-init_1       | [2021-02-24 16:23:54,814] {db.py:674} INFO - Creating tables

airflow-init_1       | INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.

airflow-init_1       | INFO  [alembic.runtime.migration] Will assume transactional DDL.

airflow-init_1       | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade  -> e3a246e0dc1, current schema

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 1507a7289a2f -> 13eb55f81627, maintain history for compatibility with earlier migrations

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 338e90f54d61, More logging into task_instance

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 338e90f54d61 -> 52d714495f0, job_id indices

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 52d714495f0 -> 502898887f84, Adding extra to Log

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 502898887f84 -> 1b38cef5b76e, add dagrun

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 1b38cef5b76e -> 2e541a1dcfed, task_duration

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 2e541a1dcfed -> 40e67319e3a9, dagrun_config

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 40e67319e3a9 -> 561833c1c74b, add password column to user

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 561833c1c74b -> 4446e08588, dagrun start end

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 4446e08588 -> bbc73705a13e, Add notification_sent column to sla_miss

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade bbc73705a13e -> bba5a7cfc896, Add a column to track the encryption state of the 'Extra' field in connection

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade bba5a7cfc896 -> 1968acfc09e3, add is_encrypted column to variable table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 1968acfc09e3 -> 2e82aab8ef20, rename user table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> 211e584da130, add TI state index

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 211e584da130 -> 64de9cddf6c9, add task fails journal table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> f2ca10b85618, add dag_stats table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade f2ca10b85618 -> 4addfa1236f1, Add fractional seconds to mysql tables

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> 8504051e801b, xcom dag task indices

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 8504051e801b -> 5e7d17757c7a, add pid field to TaskInstance

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> 127d2bf2dfa7, Add dag_id/state index on dag_run table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 127d2bf2dfa7 -> cc1e65623dc7, add max tries column to task instance

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade cc1e65623dc7 -> bdaa763e6c56, Make xcom value column a large binary

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade bdaa763e6c56 -> 947454bf1dff, add ti job_id index

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 947454bf1dff -> d2ae31099d61, Increase text size for MySQL (not relevant for other DBs' text types)

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 0e2a74e0fc9f, Add time zone awareness

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 33ae817a1ff4, kubernetes_resource_checkpointing

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 33ae817a1ff4 -> 27c6a30d7c24, kubernetes_resource_checkpointing

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 27c6a30d7c24 -> 86770d1215c0, add kubernetes scheduler uniqueness

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 86770d1215c0, 0e2a74e0fc9f -> 05f30312d566, merge heads

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 05f30312d566 -> f23433877c24, fix mysql not null constraint

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade f23433877c24 -> 856955da8476, fix sqlite foreign key

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 856955da8476 -> 9635ae0956e7, index-faskfail

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> dd25f486b8ea, add idx_log_dag

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade dd25f486b8ea -> bf00311e1990, add index to taskinstance

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> 0a2a5b66e19d, add task_reschedule table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 0a2a5b66e19d, bf00311e1990 -> 03bc53e68815, merge_heads_2

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 03bc53e68815 -> 41f5f12752f8, add superuser field

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 41f5f12752f8 -> c8ffec048a3b, add fields to dag

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade c8ffec048a3b -> dd4ecb8fbee3, Add schedule interval to dag

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade dd4ecb8fbee3 -> 939bb1e647c8, task reschedule fk on cascade delete

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 939bb1e647c8 -> 6e96a59344a4, Make TaskInstance.pool not nullable

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 6e96a59344a4 -> d38e04c12aa2, add serialized_dag table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade d38e04c12aa2 -> b3b105409875, add root_dag_id to DAG

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 6e96a59344a4 -> 74effc47d867, change datetime to datetime2(6) on MSSQL tables

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 939bb1e647c8 -> 004c1210f153, increase queue name size limit

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade c8ffec048a3b -> a56c9515abdc, Remove dag_stat table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade a56c9515abdc, 004c1210f153, 74effc47d867, b3b105409875 -> 08364691d074, Merge the four heads back together

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 08364691d074 -> fe461863935f, increase_length_for_connection_password

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade fe461863935f -> 7939bcff74ba, Add DagTags table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 7939bcff74ba -> a4c2fd67d16b, add pool_slots field to task_instance

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade a4c2fd67d16b -> 852ae6c715af, Add RenderedTaskInstanceFields table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 852ae6c715af -> 952da73b5eff, add dag_code table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 952da73b5eff -> a66efa278eea, Add Precision to execution_date in RenderedTaskInstanceFields table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade a66efa278eea -> da3f683c3a5a, Add dag_hash Column to serialized_dag table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade da3f683c3a5a -> 92c57b58940d, Create FAB Tables

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 92c57b58940d -> 03afc6b6f902, Increase length of FAB ab_view_menu.name column

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 03afc6b6f902 -> cf5dc11e79ad, drop_user_and_chart

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade cf5dc11e79ad -> bbf4a7ad0465, Remove id column from xcom

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade bbf4a7ad0465 -> b25a55525161, Increase length of pool name

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade b25a55525161 -> 3c20cacc0044, Add DagRun run_type

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 3c20cacc0044 -> 8f966b9c467a, Set conn_type as non-nullable

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 8f966b9c467a -> 8d48763f6d53, add unique constraint to conn_id

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 8d48763f6d53 -> e38be357a868, Add sensor_instance table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade e38be357a868 -> b247b1e3d1ed, Add queued by Job ID to TI

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade b247b1e3d1ed -> e1a11ece99cc, Add external executor ID to TI

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade e1a11ece99cc -> bef4f3d11e8b, Drop KubeResourceVersion and KubeWorkerId

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade bef4f3d11e8b -> 98271e7606e2, Add scheduling_decision to DagRun and DAG

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 98271e7606e2 -> 52d53670a240, fix_mssql_exec_date_rendered_task_instance_fields_for_MSSQL

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 52d53670a240 -> 364159666cbd, Add creating_job_id to DagRun table

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 364159666cbd -> 45ba3f1493b9, add-k8s-yaml-to-rendered-templates

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 45ba3f1493b9 -> 849da589634d, Prefix DAG permissions.

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 849da589634d -> 2c6edca13270, Resource based permissions.

airflow-init_1       | [2021-02-24 16:23:59,385] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.

airflow-init_1       | [2021-02-24 16:23:59,858] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | [2021-02-24 16:23:59,930] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 2c6edca13270 -> 61ec73d9401f, Add description field to connection

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 61ec73d9401f -> 64a7d6477aae, fix description field in connection to be text

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade 64a7d6477aae -> e959f08ac86c, Change field in DagCode to MEDIUMTEXT for MySql

airflow-init_1       | INFO  [alembic.runtime.migration] Running upgrade e959f08ac86c -> 82b7c48c147f, Remove can_read permission on config resource for User and Viewer role

airflow-init_1       | [2021-02-24 16:24:02,276] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.

airflow-init_1       | [2021-02-24 16:24:02,611] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | [2021-02-24 16:24:02,688] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | Upgrades done

airflow-init_1       | [2021-02-24 16:24:06,994] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | [2021-02-24 16:24:07,445] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | [2021-02-24 16:24:07,534] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.

airflow-init_1       | [2021-02-24 16:24:08,633] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | [2021-02-24 16:24:08,709] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-init_1       | Admin user airflow created

airflow-init_1       | 2.0.1

docker-apache-airflow-201_airflow-init_1 exited with code 0




    

However the Exceptions still exists and as I run docker-compose up, the webserver fails
Below is a small snippet of the many retries
docker-apache-airflow-201_airflow-webserver_1 exited with code 1

airflow-webserver_1  |

airflow-webserver_1  | [2021-02-24 16:41:37,569] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  | [2021-02-24 16:41:38,063] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  |   ____________       _____________

airflow-webserver_1  |  ____    |( )_______  /  /________      __

airflow-webserver_1  | ____  /| |_  /__  /  / __  /  __ _ | /| / /

airflow-webserver_1  | ___  ___ |  / _  /   _  / _  / / // / |/ |/ /

airflow-webserver_1  |  //  |//  //    //    //  _//|/

airflow-webserver_1  | [2021-02-24 16:41:38,109] {dagbag.py:448} INFO - Filling up the DagBag from /dev/null

airflow-webserver_1  | [2021-02-24 16:41:39,077] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  | [2021-02-24 16:41:39,151] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  | [2021-02-24 16:41:41 +0000] [23] [INFO] Starting gunicorn 19.10.0

airflow-webserver_1  | [2021-02-24 16:41:41 +0000] [23] [INFO] Listening at: http://0.0.0.0:8080 (23)

airflow-webserver_1  | [2021-02-24 16:41:41 +0000] [23] [INFO] Using worker: sync

airflow-webserver_1  | [2021-02-24 16:41:41 +0000] [27] [INFO] Booting worker with pid: 27

airflow-webserver_1  | [2021-02-24 16:41:41 +0000] [28] [INFO] Booting worker with pid: 28

airflow-webserver_1  | [2021-02-24 16:41:41 +0000] [29] [INFO] Booting worker with pid: 29

airflow-webserver_1  | [2021-02-24 16:41:41 +0000] [30] [INFO] Booting worker with pid: 30

airflow-webserver_1  | [2021-02-24 16:41:45,912] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  | [2021-02-24 16:41:45,967] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  | Running the Gunicorn Server with:

airflow-webserver_1  | Workers: 4 sync

airflow-webserver_1  | Host: 0.0.0.0:8080

airflow-webserver_1  | Timeout: 120

airflow-webserver_1  | Logfiles: - -

airflow-webserver_1  | Access Logformat:

airflow-webserver_1  | =================================================================

airflow-webserver_1  | [2021-02-24 16:41:45,994] {webserver_command.py:255} ERROR - [0 / 0] Some workers seem to have died and gunicorn did not restart them as expected

airflow-webserver_1  | [2021-02-24 16:41:46,097] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  | [2021-02-24 16:41:46,097] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

docker-apache-airflow-201_airflow-webserver_1 exited with code 137

airflow-webserver_1  |

airflow-webserver_1  | [2021-02-24 16:41:51,031] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  | [2021-02-24 16:41:51,646] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  |   ____________       _____________

airflow-webserver_1  |  ____    |( )_____  /  /________      __

airflow-webserver_1  | ____  /| |_  /__  /  / __  /  __ _ | /| / /

airflow-webserver_1  | ___  ___ |  / _  /   _  / _  / / // / |/ |/ /

airflow-webserver_1  |  //  |//  //    //    //  _/____/|__/

airflow-webserver_1  | [2021-02-24 16:41:51,703] {dagbag.py:448} INFO - Filling up the DagBag from /dev/null

airflow-webserver_1  | [2021-02-24 16:41:52,651] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  | [2021-02-24 16:41:52,729] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

airflow-webserver_1  | [2021-02-24 16:41:55 +0000] [23] [INFO] Starting gunicorn 19.10.0

airflow-webserver_1  | [2021-02-24 16:41:55 +0000] [23] [INFO] Listening at: http://0.0.0.0:8080 (23)

airflow-webserver_1  | [2021-02-24 16:41:55 +0000] [23] [INFO] Using worker: sync

airflow-webserver_1  | [2021-02-24 16:41:55 +0000] [27] [INFO] Booting worker with pid: 27

airflow-webserver_1  | [2021-02-24 16:41:55 +0000] [28] [INFO] Booting worker with pid: 28

airflow-webserver_1  | [2021-02-24 16:41:55 +0000] [29] [INFO] Booting worker with pid: 29

airflow-webserver_1  | [2021-02-24 16:41:55 +0000] [30] [INFO] Booting worker with pid: 30

airflow-webserver_1  | Running the Gunicorn Server with:

airflow-webserver_1  | Workers: 4 sync

airflow-webserver_1  | Host: 0.0.0.0:8080

airflow-webserver_1  | Timeout: 120

airflow-webserver_1  | Logfiles: - -

airflow-webserver_1  | Access Logformat:

airflow-webserver_1  | =================================================================

airflow-webserver_1  | [2021-02-24 16:42:00,595] {webserver_command.py:255} ERROR - [0 / 0] Some workers seem to have died and gunicorn did not restart them as expected

airflow-webserver_1  | [2021-02-24 16:42:01,329] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'

docker-apache-airflow-201_airflow-webserver_1 exited with code 137
          You can ignore the following line:
airflow-webserver_1 | [2021-02-24 16:42:01,329] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
The container exited because it did not have enough memory:
docker-apache-airflow-201_airflow-webserver_1 exited with code 137
Try increasing the memory: https://docs.docker.com/docker-for-mac/space/
          It worked!

I increased the memory from 2GB to 4GB and kept CPU at 8, Swap at 1GB, and Disk image size at 59.6GB (my default Docker settings)

Thank you so much for your help!
  kaxil, bastianwegge, watxaut, timsu27, arnawldo, vivekkr12, BrianWang1990, mjmare, tiagosilvayuca, and sneha-mohite reacted with hooray emoji
  cliffng, vivekkr12, chekos, tiagosilvayuca, and sneha-mohite reacted with rocket emoji
      All reactions
          I believe you ran the command:
docker compose up airflow-init
but left the following command:
docker compose up
if it still doesn't work, try:
echo -e "AIRFLOW_UID=$(id -u)" > .env
docker rm (docker ps -a -q)
docker-compose down --volumes --rm all
docker compose up airflow-init
docker compose up