Skip to content

Latest commit

 

History

History
355 lines (264 loc) · 11.4 KB

CHANGELOG.rst

File metadata and controls

355 lines (264 loc) · 11.4 KB
Raw

Changelog

2.0.3 (2024-06-06)

Patch release with bug fixes. No breaking changes.

Bug fixes

  • HTTP timeout wasn't always being correctly applied in the downstream c-questdb-client dependency.
  • request_timeout > 0 will now be enforced. This was always required, but would not error.
  • Fixed the source distribution "sdist" package: This allows the package to be installed from source via "pip install" on previously unsupported platforms (YMMV).

2.0.2 (2024-04-11)

Patch release with a performance bug fix. No breaking changes.

Bug fixes

  • Fixed the defaulting logic for auto_flush_rows parameter for HTTPS. It is now correctly set to 75000 rows by default. The old incorrect default of 600 rows was causing the sender to flush too often, impacting performance. Note that TCP, TCPS and HTTP were not affected.

Features

  • The sender now exposes the auto_flush settings as read-only properties. You can inspect the values in use with .auto_flush, .auto_flush_rows, .auto_flush_interval and .auto_flush_bytes.

2.0.1 (2024-04-03)

Patch release with bug fixes, no API changes and some documentation tweaks.

Bug fixes

  • Fixed a bug where an internal "last flushed" timestamp used by auto_flush_interval wasn't updated correctly causing the auto-flush logic to trigger after each row.
  • Removed two unnecessary debugging print() statements that were accidentally left in the code in Sender.from_conf() and Sender.from_env().

Documentation

  • Introduced the ability to optionally install pandas and pyarrow via python3 -m pip install -U questdb[dataframe] and updated the documentation to reflect this.

2.0.0 (2024-03-19)

This is a major release with new features and breaking changes.

Features

  • Support for ILP over HTTP. The sender can now send data to QuestDB via HTTP instead of TCP. This provides error feedback from the server and new features.

    conf = 'http::addr=localhost:9000;'
with Sender.from_conf(conf) as sender:
    sender.row(...)
    sender.dataframe(...)

    # Will raise `IngressError` if there is an error from the server.
    sender.flush()

  • New configuration string construction. The sender can now be also constructed from a :ref:`configuration string <sender_conf>` in addition to the constructor arguments. This allows for more flexible configuration and is the recommended way to construct a sender. The same string can also be loaded from the QDB_CLIENT_CONF environment variable. The constructor arguments have been updated and some options have changed.

  • Explicit transaction support over HTTP. A set of rows for a single table can now be committed via the sender transactionally. You can do this using a with sender.transaction('table_name') as txn: block.

    conf = 'http::addr=localhost:9000;'
with Sender.from_conf(conf) as sender:
    with sender.transaction('test_table') as txn:
        # Same arguments as the sender methods, minus the table name.
        txn.row(...)
        txn.dataframe(...)

  • A number of documentation improvements.

Breaking Changes

  • New protocol parameter in the :ref:`Sender <sender_programmatic_construction>` constructor.

    In previous version the protocol was always TCP. In this new version you must specify the protocol explicitly.

  • New auto-flush defaults. In previous versions :ref:`auto-flushing <sender_auto_flush>` was enabled by default and triggered by a maximum buffer size. In this new version auto-flushing is enabled by row count (600 rows by default) and interval (1 second by default), while auto-flushing by buffer size is disabled by default.

    The old behaviour can be still be achieved by tweaking the auto-flush settings.

    Setting

    Old default

    New default

    auto_flush_rows

    off

    600

    auto_flush_interval

    off

    1000

    auto_flush_bytes

    64512

    off

  • The at=.. argument of :func:`row <questdb.ingress.Sender.row>` and :func:`dataframe <questdb.ingress.Sender.dataframe>` methods is now mandatory. Omitting it would previously use a server-generated timestamp for the row. Now if you want a server generated timestamp, you can pass the :ref:`ServerTimestamp <sender_server_timestamp>` singleton to this parameter. _The ServerTimestamp behaviour is considered legacy._

  • The auth=(u, t, x, y) argument of the Sender constructor has now been broken up into multiple arguments: username, token, token_x, token_y.

  • The tls argument of the Sender constructor has been removed and replaced with the protocol argument. Use Protocol.Tcps (or Protocol.Https) to enable TLS. The tls values have been moved to new tls_ca and tls_roots :ref:`configuration settings <sender_conf_tls>`.

  • The net_interface argument of the Sender constructor has been renamed to bind_interface and is now only available for TCP connections.

The following example shows how to migrate to the new API.

Old questdb 1.x code

from questdb.ingress import Sender

auth = (
    'testUser1',
    '5UjEMuA0Pj5pjK8a-fa24dyIf-Es5mYny3oE_Wmus48',
    'token_x=fLKYEaoEb9lrn3nkwLDA-M_xnuFOdSt9y0Z7_vWSHLU',
    'token_y=Dt5tbS1dEDMSYfym3fgMv0B99szno-dFc1rYF9t0aac')
with Sender('localhost', 9009, auth=auth, tls=True) as sender:
    sender.row(
        'test_table',
        symbols={'sym': 'AAPL'},
        columns={'price': 100.0})  # `at=None` was defaulted for server time

Equivalent questdb 2.x code

from questdb.ingress import Sender, Protocol, ServerTimestamp

sender = Sender(
    Protocol.Tcps,
    'localhost',
    9009,
    username='testUser1',
    token='5UjEMuA0Pj5pjK8a-fa24dyIf-Es5mYny3oE_Wmus48',
    token_x='token_x=fLKYEaoEb9lrn3nkwLDA-M_xnuFOdSt9y0Z7_vWSHLU',
    token_y='token_y=Dt5tbS1dEDMSYfym3fgMv0B99szno-dFc1rYF9t0aac',
    auto_flush_rows='off',
    auto_flush_interval='off',
    auto_flush_bytes=64512)
with sender:
    sender.row(
        'test_table',
        symbols={'sym': 'AAPL'},
        columns={'price': 100.0},
        at=ServerTimestamp)

Equivalent questdb 2.x code with configuration string

from questdb.ingress import Sender

conf = (
    'tcp::addr=localhost:9009;' +
    'username=testUser1;' +
    'token=5UjEMuA0Pj5pjK8a-fa24dyIf-Es5mYny3oE_Wmus48;' +
    'token_x=token_x=fLKYEaoEb9lrn3nkwLDA-M_xnuFOdSt9y0Z7_vWSHLU;' +
    'token_y=token_y=Dt5tbS1dEDMSYfym3fgMv0B99szno-dFc1rYF9t0aac;' +
    'auto_flush_rows=off;' +
    'auto_flush_interval=off;' +
    'auto_flush_bytes=64512;')
with Sender.from_conf(conf) as sender:
    sender.row(
        'test_table',
        symbols={'sym': 'AAPL'},
        columns={'price': 100.0},
        at=ServerTimestamp)

1.2.0 (2023-11-23)

This is a minor release bringing in minor new features and a few bug fixes, without any breaking changes.

Most changes are inherited by internally upgrading to version 3.1.0 of the c-questdb-client.

Features

  • Sender(..., tls=True) now also uses the OS-provided certificate store. The tls argument can now also be set to tls='os_roots' (to only use the OS-provided certs) or tls='webpki_roots' (to only use the certs provided by the webpki-roots, i.e. the old behaviour prior to this release). The new default behaviour for tls=True is equivalent to setting tls='webpki_and_os_roots'.
  • Upgraded dependencies to newer library versions. This also includes the latest webpki-roots crate providing updated TLS CA certificate roots.
  • Various example code and documentation improvements.

Bug fixes

  • Fixed a bug where timestamp columns could not accept values before Jan 1st 1970 UTC.
  • TCP connections now enable SO_KEEPALIVE: This should ensure that connections don't drop after a period of inactivity.

1.1.0 (2023-01-04)

Features

  • High-performance ingestion of Pandas dataframes into QuestDB via ILP. We now support most Pandas column types. The logic is implemented in native code and is orders of magnitude faster than iterating the dataframe in Python and calling the Buffer.row() or Sender.row() methods: The Buffer can be written from Pandas at hundreds of MiB/s per CPU core. The new dataframe() method continues working with the auto_flush feature. See API documentation and examples for the new dataframe() method available on both the Sender and Buffer classes.
  • New TimestampNanos.now() and TimestampMicros.now() methods. These are the new recommended way of getting the current timestamp.
  • The Python GIL is now released during calls to Sender.flush() and when auto_flush is triggered. This should improve throughput when using the Sender from multiple threads.

Errata

  • In previous releases the documentation for the from_datetime() methods of the TimestampNanos and TimestampMicros types recommended calling datetime.datetime.utcnow() to get the current timestamp. This is incorrect as it will (confusinly) return object with the local timezone instead of UTC. This documentation has been corrected and now recommends calling datetime.datetime.now(tz=datetime.timezone.utc) or (more efficiently) the new TimestampNanos.now() and TimestampMicros.now() methods.

1.0.2 (2022-10-31)

Features

  • Support for Python 3.11.
  • Updated to version 2.1.1 of the c-questdb-client library:
    • Setting SO_REUSEADDR on outbound socket. This is helpful to users with large number of connections who previously ran out of outbound network ports.

1.0.1 (2022-08-16)

Features

  • As a matter of convenience, the Buffer.row method can now take None column values. This has the same semantics as skipping the column altogether. Closes #3.

Bug fixes

  • Fixed a major bug where Python int and float types were handled with 32-bit instead of 64-bit precision. This caused certain int values to be rejected and other float values to be rounded incorrectly. Closes #13.
  • Fixed a minor bug where an error auto-flush caused a second clean-up error. Closes #4.

1.0.0 (2022-07-15)

Features

  • First stable release.
  • Insert data into QuestDB via ILP.
  • Sender and Buffer APIs.
  • Authentication and TLS support.
  • Auto-flushing of buffers.

0.0.3 (2022-07-14)

Features

  • Initial set of features to connect to the database.
  • Buffer and Sender classes.
  • First release where pip install questdb should work.

0.0.1 (2022-07-08)

Features

  • First release on PyPI.