Skip to content

fix: Permission Error trying to use pb.load_data_set() with duckdb on Windows #244

Description

@matt-humphrey

Description

When running pb.load_dataset(dataset="small_table", tbl_type="duckdb") on my Windows machine, I get the following error:

File "C:\Users\00113294\Projects\pointblank\pointblank\validate.py", line 562, in load_dataset
    with tempfile.TemporaryDirectory() as tmp, ZipFile(data_path, "r") as z:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 946, in __exit__
    self.cleanup()
    ~~~~~~~~~~~~^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 950, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 930, in _rmtree
    _shutil.rmtree(name, onexc=onexc)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\shutil.py", line 790, in rmtree
    return _rmtree_unsafe(path, onexc)
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\shutil.py", line 629, in _rmtree_unsafe
    onexc(os.unlink, fullname, err)
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 905, in onexc
    _os.unlink(path)
    ~~~~~~~~~~^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\00113294\\AppData\\Local\\Temp\\tmps3x_936b\\small_table.ddb'

According to Claude:

This is a Windows-specific issue with DuckDB file locking and temporary file cleanup. The error occurs because:

  1. File Handle Not Released: When pb.load_dataset() creates a DuckDB database file in a temporary directory, the database connection isn't being properly closed before Python tries to clean up the temporary directory.
  2. Windows File Locking: Windows is more strict about file locking than Unix systems - it won't allow deletion of files that have open handles, even if the process that opened them is trying to delete them.

Development environment

  • Operating System: Windows
  • Pointblank Version: 0.11.2

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions