Source password via variable

[mkswebdev]

How hide password from manticore.conf? Can I use a variable?

docker-compose.yml:

version: "3.7"

networks:
    backend:
        driver: bridge

services:`
    manticore:
        image: manticoresearch/manticore
        restart: unless-stopped
        environment:
            - variable_sql_pass=${DB_ROOT_PASSWORD} <--- password from secret .env
        ports:
            - 127.0.0.1:9306:9306
        ulimits:
            ...
        depends_on:
            - db
        volumes:
            - ./docker/manticore:/etc/manticoresearch
        networks:
            - backend

./docker/manticore/manticore.conf:

source mysource
{
	type       = mysql
	sql_host   = db
	sql_user   = root
	sql_pass   = <---- password via variable possible?
	sql_db     = mydb
	sql_port   = 3306
}
...

[sanikolaev]

It's not possible with the default entrypoint, but it sounds like a good idea to extend the env vars functionality. Currently it supports updating searchd/common settings via env vars. Would you like to extend the functionality and make a pull request?

[mkswebdev]

Yes, please, extend the env vars functionality is a good solution.

I read the documentation and was surprised that only searchd/common settings is supported.

[sanikolaev]

Not sure when we'll have time for it, but we'll be glad to review and accept your pull request :)

[epotocko]

This is possible in v6.2.0 using the environment variable common_sql_pass (assuming a single password for all sql_pass instances).
In v6.3 the environment variable functionality has been refactored to add a common section to the config instead of replacing variables in the config. The old code is here: https://github.com/manticoresoftware/docker/blob/6.2.0/docker-entrypoint.sh#L175

[Badestrand]

In v6.3 the environment variable functionality has been refactored to add a common section to the config instead of replacing variables in the config.

What does this mean, does anyone have an example for this?

For sure there must be a solution to not have to hardcode the database credentials into the conf file?

[sanikolaev]

What does this mean, does anyone have an example for this?

It won't work because sql_pass isn't defined in the common section.

For sure there must be a solution to not have to hardcode the database credentials into the conf file?

Not right now, but I've opened a PR with a prototype of the feature: #123

Feel free to help by testing, reviewing, or improving it.

[KlimTodrik]

You can do it the following way:

1. Create dynamic config

% cat plain.conf
#!/bin/bash

cat << EOF
searchd {
    listen = 9306:mysql41
    listen = /var/run/mysqld/mysqld.sock:mysql41
    log = searchd.log
    pid_file = searchd.pid
    binlog_path =
}

source src {
    type = csvpipe
    csvpipe_command = echo "1,abc" && echo "2,abc" && echo "3,abc ${CUSTOM_VARIABLE}"
    csvpipe_field = f
}

index idx1 {
    type = plain
    source = src
    path = idx1
    stored_fields = f
}
EOF

2) Run the Docker container

docker run -d \
-e CUSTOM_VARIABLE=hello \
--name manticore \
-v $(pwd)/plain.conf:/etc/manticoresearch/manticore.conf \
manticoresearch/manticore:dev

3) Verify the generated config

% docker exec manticore cat /etc/manticoresearch/manticore.conf.debug

You should see that ${CUSTOM_VARIABLE} was substituted:

csvpipe_command = echo "1,abc" && echo "2,abc" && echo "3,abc hello"

[sanikolaev]

@KlimTodrik let’s include this in the README so users know this trick exists.

[KlimTodrik]

Done b9c9a11