Skip to content

pgcli crashes when using use directive #1484

Open
#1628
Open
pgcli crashes when using use directive#1484
#1628
Labels
bug

Description

@dlangille
dlangille
opened on Nov 24, 2024

Description

pgcli crashes when using use directive

Your environment

PostgreSQL 16.5
pgcli-4.1.0
Python 3.11

  • FreeBSD 14.1
  • 4.1.0
  • zsh: command not found: pip (I install from package).
dvl@/tmp:samdrucker> use freshports.dvl
You are now connected to database "freshports.dvl" as user "dvl"
Time: 0.022s
Traceback (most recent call last):
  File "/usr/local/bin/pgcli", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
^[[25;1R    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/main.py", line 1640, in cli
    pgcli.run_cli()
  File "/usr/local/lib/python3.11/site-packages/pgcli/main.py", line 971, in run_cli
    text = self.prompt_app.prompt()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1035, in prompt
    return self.app.run(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 1002, in run
    return asyncio.run(coro)
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 886, in run_async
Exception in thread completion_refresh:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    return await _run_async(f)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 739, in _run_async
    self.run()
  File "/usr/local/lib/python3.11/threading.py", line 982, in run
    self._redraw()
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 543, in _redraw
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.11/site-packages/pgcli/completion_refresher.py", line 67, in _bg_refresh
    self.context.copy().run(run_in_context)
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 526, in run_in_context
    refresher(completer, executor)
  File "/usr/local/lib/python3.11/site-packages/pgcli/completion_refresher.py", line 108, in refresh_schemata
    self.renderer.render(self, self.layout)
    completer.set_search_path(executor.search_path())
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/renderer.py", line 641, in render
  File "/usr/local/lib/python3.11/site-packages/pgcli/pgcompleter.py", line 328, in set_search_path
    layout.container.preferred_height(size.columns, size.rows).preferred,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 318, in preferred_height
    self.search_path = self.escaped_names(search_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/pgcompleter.py", line 176, in escaped_names
    dimensions = [
                 ^
    return [self.escape_name(name) for name in names]
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 319, in <listcomp>
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/pgcompleter.py", line 176, in <listcomp>
    c.preferred_height(width, max_available_height) for c in self._all_children
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    return [self.escape_name(name) for name in names]
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 786, in preferred_height
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/pgcompleter.py", line 157, in escape_name
    return self.content.preferred_height(width, max_available_height)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (not self.name_pattern.match(name))
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 318, in preferred_height
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot use a string pattern on a bytes-like object
    dimensions = [
                 ^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 319, in <listcomp>
    c.preferred_height(width, max_available_height) for c in self._all_children
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 2623, in preferred_height
    if self.filter():
       ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/filters/base.py", line 254, in __call__
    return self.func()
           ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 161, in has_before_fragments
    for fragment, char, *_ in get_prompt_text():
                              ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1280, in _get_prompt
    return to_formatted_text(self.message, style="class:prompt")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/formatted_text/base.py", line 82, in to_formatted_text
    return to_formatted_text(value(), style=style)
                             ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/main.py", line 1043, in get_message
    prompt = self.get_prompt(prompt_format)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/main.py", line 1314, in get_prompt
    string = string.replace("\\H", self.pgexecute.host or "(none)")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: replace() argument 2 must be str, not bytes
[20:41 pg03 dvl ~] % pgcli samdrucker
Server: PostgreSQL 16.4
Version: 4.1.0
Home: http://pgcli.com
dvl@/tmp:samdrucker> use fres
connection failed: connection to server on socket "/tmp/.s.PGSQL.5432" failed: FATAL:  database "fres" does not exist
Previous connection kept
You are now connected to database "samdrucker" as user "dvl"
Time: 0.009s
dvl@/tmp:samdrucker> use freshports.freshports.dvl
connection failed: connection to server on socket "/tmp/.s.PGSQL.5432" failed: FATAL:  database "freshports.freshports.dvl" does not exist
Previous connection kept
You are now connected to database "samdrucker" as user "dvl"
Time: 0.012s
dvl@/tmp:samdrucker> use freshports.dvl
You are now connected to database "freshports.dvl" as user "dvl"
Time: 0.021s
Traceback (most recent call last):
  File "/usr/local/bin/pgcli", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
^[[50;1R    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/main.py", line 1640, in cli
    pgcli.run_cli()
  File "/usr/local/lib/python3.11/site-packages/pgcli/main.py", line 971, in run_cli
    text = self.prompt_app.prompt()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1035, in prompt
    return self.app.run(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 1002, in run
    return asyncio.run(coro)
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
Exception in thread completion_refresh:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    self.run()
  File "/usr/local/lib/python3.11/threading.py", line 982, in run
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 886, in run_async
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.11/site-packages/pgcli/completion_refresher.py", line 67, in _bg_refresh
    return await _run_async(f)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 739, in _run_async
    refresher(completer, executor)
  File "/usr/local/lib/python3.11/site-packages/pgcli/completion_refresher.py", line 108, in refresh_schemata
    completer.set_search_path(executor.search_path())
    self._redraw()
  File "/usr/local/lib/python3.11/site-packages/pgcli/pgcompleter.py", line 328, in set_search_path
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 543, in _redraw
    self.search_path = self.escaped_names(search_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/pgcompleter.py", line 176, in escaped_names
    self.context.copy().run(run_in_context)
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 526, in run_in_context
    return [self.escape_name(name) for name in names]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/pgcompleter.py", line 176, in <listcomp>
    return [self.escape_name(name) for name in names]
    self.renderer.render(self, self.layout)
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/renderer.py", line 641, in render
  File "/usr/local/lib/python3.11/site-packages/pgcli/pgcompleter.py", line 157, in escape_name
    (not self.name_pattern.match(name))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    layout.container.preferred_height(size.columns, size.rows).preferred,
TypeError: cannot use a string pattern on a bytes-like object
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 318, in preferred_height
    dimensions = [
                 ^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 319, in <listcomp>
    c.preferred_height(width, max_available_height) for c in self._all_children
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 786, in preferred_height
    return self.content.preferred_height(width, max_available_height)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 318, in preferred_height
    dimensions = [
                 ^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 319, in <listcomp>
    c.preferred_height(width, max_available_height) for c in self._all_children
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/layout/containers.py", line 2623, in preferred_height
    if self.filter():
       ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/filters/base.py", line 254, in __call__
    return self.func()
           ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 161, in has_before_fragments
    for fragment, char, *_ in get_prompt_text():
                              ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1280, in _get_prompt
    return to_formatted_text(self.message, style="class:prompt")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/formatted_text/base.py", line 82, in to_formatted_text
    return to_formatted_text(value(), style=style)
                             ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/main.py", line 1043, in get_message
    prompt = self.get_prompt(prompt_format)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pgcli/main.py", line 1314, in get_prompt
    string = string.replace("\\H", self.pgexecute.host or "(none)")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: replace() argument 2 must be str, not bytes

Activity

  1. dbaty commented on Nov 25, 2024

    @dbaty
    dbaty
    on Nov 25, 2024
    Member

    The error does not seem to be related to use, as it happens on this line:

    string = string.replace("\\H", self.pgexecute.host or "(none)")

    It looks like self.pgexecute.host here is a bytes object, whereas we expect it to be a string. I cannot reproduce (yet).

    As a temporary workaround, you could tweak your pgcli prompt and remove this \H. It's possible that other bindings fail the same (\h, etc.), so you may have to remove these, too.

  2. ebishop-retool commented on Jun 13, 2025

    @ebishop-retool
    ebishop-retool
    on Jun 13, 2025

    Btw, I'm seeing the same error message and almost exactly the same stacktrace, but for me it happens immediately when I run pgcli -p $MY_PORT $MY_DB $MY_USER. In other words, I don't get a chance to enter use or any other command. I'm also on v4.3.0, but on MacOS 15.5.

    The difference in the stacktrace for me is that the Python it's using is a Python 3.13 installation that seems to be packaged with pgcli in Homebrew, so everywhere that /usr/local/lib/python3.11 appears in the stacktrace above, I have /opt/homebrew/Cellar/pgcli/4.3.0/libexec/lib/python3.13/ instead.

  3. dbaty
    added
    bug
    on May 31, 2026

  4. DiegoDAF commented on Sep 1, 2026

    @DiegoDAF
    DiegoDAF
    on Sep 1, 2026
    Contributor

    Hi,

    Thanks for the report. I can reproduce this and I think the three symptoms
    (TypeError on the prompt, the b'...'-prefixed timezone message, and the
    completion_refresh thread dying in parse_defaults) all share one root
    cause.

    When the client encoding cannot decode a text value (most commonly
    SQL_ASCII), psycopg returns the text columns to us as raw bytes instead of
    str. A few code paths were passing those values through unguarded:

    • The socket directory query result was used to build the prompt, so a Unix
      socket connection raised TypeError: expected str, not bytes.
    • The show timezone result was printed in the startup message, showing a
      b'...'-prefixed value.
    • The function metadata query (used by the background completion refresh)
      selected text columns such as arg_defaults; parsing those bytes with
      parse_defaults() crashed the refresh thread with
      TypeError: can only concatenate str (not "int") to str.

    The fix is to decode those values defensively when they come back as bytes
    (same guard we already use in the completion metadata fix for #1405). It is a
    small, self-contained change: a _decode_if_bytes() helper plus decoding the
    socket directory, the timezone, and the function metadata rows. I have
    reproduced it against a real SQL_ASCII cluster (3396 catalog functions, all
    metadata fields come back as plain str after the fix) and added unit tests.

    A small PR with tests is coming. I'll link it here once it is in.

    its the same for #1518

  5. DiegoDAF
    added a commit that references this issue on Sep 1, 2026

    Defensively decode SQL_ASCII text columns in prompt, timezone, and

    b9c9c56

  6. DiegoDAF commented on Sep 1, 2026

    @DiegoDAF
    DiegoDAF
    on Sep 1, 2026
    Contributor

    Hi,

    Thanks for the report. I can reproduce this and I think the three symptoms (TypeError on the prompt, the b'...'-prefixed timezone message, and the completion_refresh thread dying in parse_defaults) all share one root cause.

    When the client encoding cannot decode a text value (most commonly SQL_ASCII), psycopg returns the text columns to us as raw bytes instead of str. A few code paths were passing those values through unguarded:

    • The socket directory query result was used to build the prompt, so a Unix socket connection raised TypeError: expected str, not bytes.
    • The show timezone result was printed in the startup message, showing a b'...'-prefixed value.
    • The function metadata query (used by the background completion refresh) selected text columns such as arg_defaults; parsing those bytes with parse_defaults() crashed the refresh thread with TypeError: can only concatenate str (not "int") to str.

    The fix is to decode those values defensively when they come back as bytes (same guard we already use in the completion metadata fix for #1405). It is a small, self-contained change: a _decode_if_bytes() helper plus decoding the socket directory, the timezone, and the function metadata rows. I have reproduced it against a real SQL_ASCII cluster (3396 catalog functions, all metadata fields come back as plain str after the fix) and added unit tests.

    The fix is up as #1628.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions