Here's an issue I came across. I already found a solution myself that I'll show below.
I ran the docker compose up -d command for the default docker-compose-gpu.yml file on Windows/Docker Desktop. I received the following error:
decoding failed due to the following error(s): 'services[llama-cpp-server].command' invalid command line string
I asked chatgpt about it. And it gave me an explanation and a very simple solution.
Docker Compose expects the command to be either a string or a list of arguments. Using > (a folded scalar) can cause issues because it turns everything into a single string with line breaks, and some Docker Compose versions can’t parse that properly as a command line.
The solution was to change the specific part in the yml file, to something like this:
command: [ "-m", "/models/${ORPHEUS_MODEL_NAME}", "--port", "5006", "--host", "0.0.0.0", "--n-gpu-layers", "29", "--ctx-size", "${ORPHEUS_MAX_TOKENS}", "--n-predict", "${ORPHEUS_MAX_TOKENS}", "--rope-scaling", "linear" ]
My docker instance is running fine now, so I don't need any help on this. I just thought I'd let you all know about it.
Gr
Here's an issue I came across. I already found a solution myself that I'll show below.
I ran the docker compose up -d command for the default docker-compose-gpu.yml file on Windows/Docker Desktop. I received the following error:
decoding failed due to the following error(s): 'services[llama-cpp-server].command' invalid command line stringI asked chatgpt about it. And it gave me an explanation and a very simple solution.
Docker Compose expects the command to be either a string or a list of arguments. Using > (a folded scalar) can cause issues because it turns everything into a single string with line breaks, and some Docker Compose versions can’t parse that properly as a command line.
The solution was to change the specific part in the yml file, to something like this:
command: [ "-m", "/models/${ORPHEUS_MODEL_NAME}", "--port", "5006", "--host", "0.0.0.0", "--n-gpu-layers", "29", "--ctx-size", "${ORPHEUS_MAX_TOKENS}", "--n-predict", "${ORPHEUS_MAX_TOKENS}", "--rope-scaling", "linear" ]My docker instance is running fine now, so I don't need any help on this. I just thought I'd let you all know about it.
Gr