Heroku-Redis SSL Issues

[akcode47]

The following redis.ConnectionPool.from_url(config.REDIS_URL) works with Heroku's REDIS_URL, but due to their TLS changes, I was unable to connect using create_pool(RedisSettings().from_dsn(config.REDIS_URL)) which led to the following SSLCertVerificationError error:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1006)

import redis.asyncio as redis
pool = redis.ConnectionPool.from_url(config.REDIS_URL)

No matter how I pass in ssl=True to the RedisSettings class and set ssl_cert_reqs=none, I get ConnectionErrors, whereas with redis.asyncio, I can establish the pool/client.

While digging around the ARQ code, I saw that the ArqRedis class allows you to pass a pool directly to the pool_or_conn=pool parameter, which works as a workaround for me.

[Graeme22]

This code works for me:

# create job queue
redis_settings = RedisSettings.from_dsn(settings.REDIS_URL)
redis_settings.ssl_cert_reqs = "none"

[akcode47]

This code works for me:

# create job queue
redis_settings = RedisSettings.from_dsn(settings.REDIS_URL)
redis_settings.ssl_cert_reqs = "none"

Thank you - my issue stemmed from the RedisSettings object initialization and modification.

Previously, I attempted to set ssl_cert_reqs directly in the constructor and immediately chained the from_dsn() class method, which didn't work because from_dsn() is designed to parse and return a new instance of RedisSettings based on the DSN, which overrides any attributes set during object construction.

Won't work:

from arq.connections import RedisSettings, create_pool
async def example():
    redis_settings = RedisSettings(ssl_cert_reqs = "none").from_dsn(config.REDIS_URL)
    return await create_pool(redis_settings)

In the following working version, the RedisSettings class is initialized with the from_dsn() method, and the instance's ssl_cert_reqs property is set to the string value 'none':

from arq.connections import RedisSettings, create_pool
async def example():
    redis_settings = RedisSettings.from_dsn(config.REDIS_URL)
    redis_settings.ssl_cert_reqs = "none"
    return await create_pool(redis_settings)

Not sure if this is worth mentioning in the docs or if it makes sense to enhance the from_dsn() class method to parse the query parameters and set the ssl_cert_reqs property accordingly. Something like the following:

conf = urlparse(dsn)
query_params = parse_qs(conf.query)

# Extract SSL settings from query parameters with a fallback to defaults
ssl_cert_reqs = query_params.get('ssl_cert_reqs', ['required'])[0]

[Graeme22]

No problem! I agree that the constructor should accept and pass on the kwargs. However considering there's an entire rewrite in progress I guess it's not super high priority right now!

[samuelcolvin]

Thanks for reporting.

No problem! I agree that the constructor should accept and pass on the kwargs. However considering there's an entire rewrite in progress I guess it's not super high priority right now!

In some senses, now is a good time to fix this - if we merge a fix and more importantly tests, we'll have to support that behavior after the rewrite.