Skip to content

Vortex Vectors#4954

Closed
gatesn wants to merge 3 commits into
developfrom
ngates/vector
Closed

Vortex Vectors#4954
gatesn wants to merge 3 commits into
developfrom
ngates/vector

Conversation

@gatesn
Copy link
Copy Markdown
Contributor

@gatesn gatesn commented Oct 15, 2025

Vectors are the first part of the move to making Vortex arrays logical plans. They will eventually largely subsume the physical functionality of the canonical arrays (leaving behind logical functionality), as well as array builders.

They follow a similar pattern to Tokio Bytes and vortex-buffer with a Vector and VectorMut that can be converted between each other with zero-copy.

@gatesn gatesn marked this pull request as draft October 15, 2025 18:06
@gatesn gatesn added the changelog/feature A new feature label Oct 15, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented Oct 15, 2025

Codecov Report

❌ Patch coverage is 40.13841% with 519 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.82%. Comparing base (5348219) to head (4480153).

Files with missing lines Patch % Lines
vortex-vector/src/primitive/vector_mut.rs 0.00% 81 Missing ⚠️
vortex-vector/src/bool/vector_mut.rs 0.00% 64 Missing ⚠️
vortex-vector/src/primitive/generic_mut.rs 0.00% 61 Missing ⚠️
vortex-vector/src/primitive/vector.rs 0.00% 53 Missing ⚠️
vortex-vector/src/vector/ops.rs 0.00% 52 Missing ⚠️
vortex-mask/src/mask_mut.rs 89.42% 35 Missing ⚠️
vortex-vector/src/null/vector_mut.rs 0.00% 35 Missing ⚠️
vortex-vector/src/bool/vector.rs 0.00% 34 Missing ⚠️
vortex-vector/src/primitive/generic.rs 0.00% 28 Missing ⚠️
vortex-vector/src/null/vector.rs 0.00% 17 Missing ⚠️
... and 8 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@connortsui20 connortsui20 force-pushed the ngates/vector branch 3 times, most recently from c9f50a0 to 74ffee3 Compare October 20, 2025 16:07
@connortsui20 connortsui20 force-pushed the ngates/vector branch 2 times, most recently from 060a04e to bacb67b Compare October 20, 2025 18:27
gatesn and others added 3 commits October 20, 2025 14:30
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20 connortsui20 marked this pull request as ready for review October 20, 2025 18:31
@connortsui20
Copy link
Copy Markdown
Contributor

Since I need nick to review this I'll open a new PR.

gatesn added a commit that referenced this pull request Oct 21, 2025
Tracking Issue: #4492

Continuation of #4954

This is mostly boilerplate code with a bit more documentation than
normal.

There are still a few bugs in the implementation (documented by TODOs),
but I would like this PR to just focus on the structure of the crate
rather than the logic.

Also add logic in other crates that was missing before.

---

The general structure of the crate is as such:

```
vortex-vector
├── Cargo.toml
└── src
    ├── <type>
    │   ├── mod.rs
    │   ├── vector.rs
    │   └── vector_mut.rs
    ├── lib.rs
    ├── macros.rs
    ├── ops.rs
    ├── private.rs
    ├── vector.rs
    └── vector_mut.rs
```

The exceptions to this structure will be `primitive`, which needs a
generic type as well (so there will be extra files), a `varbin` module
that holds both `binary` and `string` modules, and probably the 2 list
types might share a lot of code (we'll see once I get there).

Again, a lot of the logic here is not super clean and some of it is
wrong, but I'd like to get this merged first before I fix those.

---------

Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Co-authored-by: Nicholas Gates <nick@nickgates.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants