Release review 2024-06-29

Release https://gitlab.pg.innopolis.university/meow/ragnar/-/releases/1.0.1 review:

It is nice that you have releases now. Consider providing short description in each release what was changed/fixed. Checkout Gitlab's doc about https://docs.gitlab.com/ee/user/project/releases/ Add appropriate features to your releases. For example it would be nice to associate milestones with releases.

Review process

1. Downloaded and unpacked release 1.0.1.
2. Opened README.md and followed instructions from step 4.
3. Run docker build . -t ragnar-llm command. Got the same error as before:

❯ docker build . -t ragnar-llm
[+] Building 32.7s (8/13)                                                                            docker:default
 => [internal] load .dockerignore                                                                              0.1s
 => => transferring context: 61B                                                                               0.0s
 => [internal] load build definition from Dockerfile                                                           0.1s
 => => transferring dockerfile: 1.01kB                                                                         0.0s
 => [internal] load metadata for docker.io/library/python:3.11                                                 0.0s
 => [1/9] FROM docker.io/library/python:3.11                                                                   0.0s
 => [internal] load build context                                                                              0.0s
 => => transferring context: 257.17kB                                                                          0.0s
 => CACHED [2/9] RUN mkdir -p /code/ragnar/models                                                              0.0s
 => CACHED [3/9] RUN mkdir -p /code/ragnar/vector_db                                                           0.0s
 => ERROR [4/9] RUN wget -nv "https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF/resolve/mai  32.6s
------
 > [4/9] RUN wget -nv "https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF/resolve/main/capybarahermes-2.5-mistral-7b.Q4_K_M.gguf?download=true" -O /code/ragnar/models/capybarahermes-2.5-mistral-7b.Q4_K_M.gguf:
32.54 wget: unable to resolve host address ‘huggingface.co’
------
Dockerfile:22
--------------------
  20 |     RUN mkdir -p /code/ragnar/vector_db
  21 |     # Download LLM:
  22 | >>> RUN wget -nv "https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF/resolve/main/capybarahermes-2.5-mistral-7b.Q4_K_M.gguf?download=true" -O /code/ragnar/models/capybarahermes-2.5-mistral-7b.Q4_K_M.gguf
  23 |     # System deps:
  24 |     RUN curl -sSL https://install.python-poetry.org | python3
--------------------
ERROR: failed to solve: process "/bin/sh -c wget -nv \"https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF/resolve/main/capybarahermes-2.5-mistral-7b.Q4_K_M.gguf?download=true\" -O /code/ragnar/models/capybarahermes-2.5-mistral-7b.Q4_K_M.gguf" did not complete successfully: exit code: 4

I downloaded the model separately using wget command provided in the Dockerfile. It worked fine. I modified Dockerfile to copy the model instead of downloading it. The build process went beyond that point but got stuck on another step:

docker build . -t ragnar-llm
[+] Building 73.7s (12/13)                                                                                    docker:default
 => [internal] load build definition from Dockerfile                                                                    0.0s
 => => transferring dockerfile: 1.13kB                                                                                  0.0s
 => [internal] load .dockerignore                                                                                       0.0s
 => => transferring context: 61B                                                                                        0.0s
 => [internal] load metadata for docker.io/library/python:3.11                                                          0.0s
 => [1/9] FROM docker.io/library/python:3.11                                                                            0.0s
 => [internal] load build context                                                                                      24.2s
 => => transferring context: 4.37GB                                                                                    24.1s
 => CACHED [2/9] RUN mkdir -p /code/ragnar/models                                                                       0.0s
 => CACHED [3/9] RUN mkdir -p /code/ragnar/vector_db                                                                    0.0s
 => [4/9] COPY models/capybarahermes-2.5-mistral-7b.Q4_K_M.gguf /code/ragnar/models/capybarahermes-2.5-mistral-7b.Q4_  13.0s
 => [5/9] RUN curl -sSL https://install.python-poetry.org | python3                                                    35.0s
 => [6/9] WORKDIR /code                                                                                                 0.0s
 => [7/9] COPY poetry.lock pyproject.toml /code/                                                                        0.0s
 => ERROR [8/9] RUN poetry install --no-interaction --no-ansi                                                           0.5s
------
 > [8/9] RUN poetry install --no-interaction --no-ansi:
0.498 /bin/sh: 1: poetry: not found
------
Dockerfile:31
--------------------
  29 |     WORKDIR /code
  30 |     COPY poetry.lock pyproject.toml /code/
  31 | >>> RUN poetry install --no-interaction --no-ansi
  32 |     COPY ./ragnar /code/ragnar/
  33 |     EXPOSE 8020
--------------------
ERROR: failed to solve: process "/bin/sh -c poetry install --no-interaction --no-ansi" did not complete successfully: exit code: 127

Improvement proposal

Since models are so big we should have a folder that we mount in the container. We should add all the models in the folder to .gitignore so they are not pushed to the repository. In the README.md we can add an extra step that requires downloading models before running the container.