Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sqlite3.OperationalError: database is locked #128

Open
Zhankin opened this issue Dec 28, 2024 · 3 comments
Open

sqlite3.OperationalError: database is locked #128

Zhankin opened this issue Dec 28, 2024 · 3 comments

Comments

@Zhankin
Copy link

Zhankin commented Dec 28, 2024

Some times in logs see this error:

INFO:     192.168.1.91:63898 - "POST /api/downloads HTTP/1.1" 200 OK
INFO:     192.168.1.91:63898 - "GET /api/downloads/stats HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        self.scope, self.receive, self.send
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/usr/src/app/ydl_server/views.py", line 88, in api_queue_size
    jobs = db.get_jobs(app_config["ydl_server"].get("max_log_entries", 100))
  File "/usr/src/app/ydl_server/db.py", line 405, in get_jobs
    cursor.execute(
    ~~~~~~~~~~~~~~^
        """
        ^^^
    ...<6 lines>...
        (str(limit),),
        ^^^^^^^^^^^^^^
    )
    ^
sqlite3.OperationalError: database is locked
INFO:     127.0.0.1:41784 - "GET /api/info HTTP/1.1" 200 OK

Powered by [yt-dlp](https://github.com/yt-dlp/yt-dlp#readme) version 2024.12.13. Code & issues on [GitHub](https://github.com/nbr23/youtube-dl-server). Rev [769ff82](https://github.com/nbr23/youtube-dl-server/commit/769ff82)

docker compose:

services:
  youtube-dl:
    image: "nbr23/youtube-dl-server:latest"
    ports:
      - 8080:8080
    volumes:
      - /mnt/storage/media/youtube-dl:/youtube-dl
      - ./config.yml:/app_config/config.yml:ro 
    restart: always
@nbr23
Copy link
Owner

nbr23 commented Dec 28, 2024

hi!
sounds like the sqlite db is locked by another process, you can try identifying it and killing it as explained here: https://stackoverflow.com/a/3481134

@Zhankin
Copy link
Author

Zhankin commented Dec 28, 2024

With a clean setup, I'm getting the same error. I cleaned all db files and cache

Using configuration file /app_config/config.yml
Migrating database from version -1
No jobs table found, creating
Using yt-dlp module
Started dl worker 0
Started dl worker 1
Started download threads
Started jobs manager thread
INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
INFO:     192.168.1.91:52822 - "GET /api/downloads/stats HTTP/1.1" 200 OK
INFO:     192.168.1.91:52823 - "GET /api/info HTTP/1.1" 200 OK
INFO:     192.168.1.91:52824 - "GET /api/extractors HTTP/1.1" 200 OK
INFO:     192.168.1.91:52825 - "GET /api/formats HTTP/1.1" 200 OK
INFO:     192.168.1.91:52822 - "GET /api/info HTTP/1.1" 200 OK
INFO:     192.168.1.91:52823 - "GET /api/info HTTP/1.1" 200 OK
INFO:     127.0.0.1:47532 - "GET /api/info HTTP/1.1" 200 OK
INFO:     192.168.1.91:52828 - "GET /api/downloads/stats HTTP/1.1" 200 OK
INFO:     192.168.1.91:52830 - "GET /api/downloads/stats HTTP/1.1" 200 OK
INFO:     192.168.1.91:52835 - "GET /api/downloads/stats HTTP/1.1" 200 OK
INFO:     192.168.1.91:52836 - "GET /api/downloads/stats HTTP/1.1" 200 OK
INFO:     192.168.1.91:52838 - "GET /api/downloads/stats HTTP/1.1" 200 OK
Added url https://youtu.be/V64ab_y2QNE?si=k18iRlan2r4xKSAQ to the download queue
INFO:     192.168.1.91:52839 - "POST /api/downloads HTTP/1.1" 200 OK
INFO:     127.0.0.1:41462 - "GET /api/info HTTP/1.1" 200 OK
INFO:     192.168.1.91:52839 - "GET /api/downloads HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        self.scope, self.receive, self.send
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/usr/src/app/ydl_server/views.py", line 108, in api_logs
    db.get_jobs_with_logs(
    ~~~~~~~~~~~~~~~~~~~~~^
        app_config["ydl_server"].get("max_log_entries", 100),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        request.query_params.get("status", None)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        )
        ^
  File "/usr/src/app/ydl_server/db.py", line 351, in get_jobs_with_logs
    cursor.execute(
    ~~~~~~~~~~~~~~^
        """
        ^^^
    ...<6 lines>...
        (str(limit),),
        ^^^^^^^^^^^^^^
    )
    ^
sqlite3.OperationalError: database is locked

@nbr23
Copy link
Owner

nbr23 commented Jan 2, 2025

I tried a few things but was not able to do it locally... Is your db on a volume on an nfs or some shared volume maybe? Does this consistently happen, or just once in a while?
I pushed an update trying to be more diligent about closing db cursors, I'm not sure if that'll help...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nbr23 @Zhankin