Skip to content

C API scan examples#7564

Open
myrrc wants to merge 1 commit intodevelopfrom
myrrc/scan-api-docs
Open

C API scan examples#7564
myrrc wants to merge 1 commit intodevelopfrom
myrrc/scan-api-docs

Conversation

@myrrc
Copy link
Copy Markdown
Contributor

@myrrc myrrc commented Apr 20, 2026

  • Add C scan api examples: dtype print, single-thread scan to arrow, multi-thread scan.
  • Remove vx_data_source_row_count in favour of vx_estimate.
  • Remove old C example.
  • Allow reading local relative paths in Datasource

@myrrc myrrc requested a review from 0ax1 April 20, 2026 11:17
@myrrc myrrc added the changelog/docs A docs change label Apr 20, 2026
@myrrc myrrc enabled auto-merge (squash) April 20, 2026 11:20
Comment thread vortex-ffi/CMakeLists.txt
Comment thread vortex-ffi/examples/scan.c Outdated
Comment thread vortex-ffi/examples/scan.c Outdated
Comment thread vortex-ffi/examples/scan.c
Comment thread vortex-ffi/examples/scan.c
Comment thread vortex-ffi/README.md Outdated
Comment thread vortex-ffi/examples/scan.c
Comment thread vortex-ffi/examples/scan.c Outdated
Comment thread vortex-ffi/examples/scan.c Outdated
Comment thread vortex-ffi/examples/scan.c Outdated
Comment thread vortex-ffi/examples/scan.c
@myrrc myrrc force-pushed the myrrc/scan-api-docs branch from 6284347 to 8e70414 Compare April 22, 2026 13:31
@myrrc myrrc changed the title C API scan example C API scan examples Apr 22, 2026
@myrrc myrrc force-pushed the myrrc/scan-api-docs branch from e675e19 to f143c23 Compare April 22, 2026 15:47
@myrrc myrrc requested a review from 0ax1 April 22, 2026 15:53
@myrrc myrrc force-pushed the myrrc/scan-api-docs branch 2 times, most recently from 2ebedaf to ba77e1f Compare April 23, 2026 12:04
Signed-off-by: Mikhail Kot <to@myrrc.dev>
@myrrc myrrc force-pushed the myrrc/scan-api-docs branch from ba77e1f to 015ff48 Compare April 23, 2026 12:07
@myrrc myrrc disabled auto-merge April 23, 2026 12:51
Comment thread .github/workflows/rust-instrumented.yml
@0ax1
Copy link
Copy Markdown
Contributor

0ax1 commented Apr 23, 2026

@claude in depth review

@github-actions
Copy link
Copy Markdown
Contributor

Claude review automation is disabled for pull requests that modify .github/ files.

Why:

  • workflow and action files are part of the automation policy
  • this review workflow refuses to evaluate automation changes from the same PR

Ask a human reviewer to inspect workflow changes directly.

Comment thread vortex-ffi/README.md
@myrrc myrrc requested a review from 0ax1 April 23, 2026 14:18
}
}

int main(int argc, char *argv[]) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add explicit return 0 for C main.

return array;
}

int main(int argc, char *argv[]) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add explicit return 0 for C main.

VX_CARD_ESTIMATE = 1,
VX_CARD_MAXIMUM = 2,
} vx_cardinality;
/**
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a diagnostic to the header that this API is unstable? I guess in the end we'd want ABI stability here. We don't have to address this in this PR but keep in mind that basically anything that lands in Vortex, will be tried out by others so it's good to be explicit on what's stable and what's unstable.

}
None => {
rc.cardinality = vx_cardinality::VX_CARD_UNKNOWN;
rc.r#type = vx_estimate_type::VX_ESTIMATE_UNKNOWN;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This requires the call site to init the row count which not all call sites in this PR do.

// Caller can use partition estimates to schedule worker threads.
print_estimate("Partition count", &partition_estimate);
if (threads == 0 && partition_estimate.type != VX_ESTIMATE_UNKNOWN) {
threads = partition_estimate.estimate;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the user didn't pass a thread count and the estimate is unknown the thread count stays 0.

const vx_array *height_array =
vx_array_new_primitive(PTYPE_U16, height_buffer, SAMPLE_ROWS, &validity, &error);
if (error != NULL) {
print_error("Error adding age array field to root array", error);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error adding height..


vx_error *error = NULL;
const vx_array *age_array = vx_array_new_primitive(PTYPE_U8, age_buffer, SAMPLE_ROWS, &validity, &error);
if (error != NULL) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error handling looks redundant in this fn, could extract into a separate fn.

}

struct scan_thread_info {
pthread_t thread_id;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're using pthread_t here but later assign ints.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/docs A docs change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants