Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a way to directly fetch a batch rather than a row? #115

Open
acking-you opened this issue Jul 30, 2024 · 1 comment
Open

Is there a way to directly fetch a batch rather than a row? #115

acking-you opened this issue Jul 30, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@acking-you
Copy link

I've noticed that most examples return data in a row-by-row format.
Is there a way to get it in batches instead?🤔
For example, something like this process could retrieve an arrow formatted batch:

use clickhouse::Client;
use tokio;
use arrow::ipc::reader::StreamReader;
use std::io::Cursor;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Creating a ClickHouse Client
    let client = Client::default()
        .with_url("http://localhost:8123")
        .with_database("default")
        .with_user("default")
        .with_password("");

    // Execute a query and get the data in Arrow IPC format
    let response = client.query("SELECT id, name, value FROM my_table FORMAT ArrowStream")
        .fetch_raw()
        .await?;

    // Convert response data to Arrow StreamReader
    let cursor = Cursor::new(response);
    let mut reader = StreamReader::try_new(cursor)?;

    // Processing Arrow batch data
    while let Some(batch) = reader.next() {
        let batch = batch?;
        println!("{:?}", batch);
    }

    println!("Arrow IPC read completed.");

    Ok(())
}
@loyd
Copy link
Collaborator

loyd commented Aug 4, 2024

Now there is no way to do it, but it is more likely the crate will provide query_raw() for such cases

@loyd loyd added the enhancement New feature or request label Aug 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants