• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

vortex-data / vortex / 16597359372

29 Jul 2025 01:22PM UTC coverage: 82.703% (+0.4%) from 82.294%
16597359372

push

github

web-flow
[chore] conformance testing for TakeFn (#4043)

FLUP to #4041 

⏺ Summary of Take Conformance Test Implementation

I successfully implemented a comprehensive conformance test suite for
the take compute function across the Vortex codebase, following the
pattern
  established for the filter conformance tests.

  1. Created Take Conformance Test Harness

- Added /vortex-array/src/compute/conformance/take.rs with a flexible
test suite
  - Implemented test functions covering various scenarios:
    - test_take_all - Take all elements in order
    - test_take_none - Take no elements (empty indices)
    - test_take_selective - Take every other element
    - test_take_first_and_last - Take first and last elements
    - test_take_with_nullable_indices - Take with nullable indices
    - test_take_repeated_indices - Take same element multiple times
    - test_empty_indices - Take with empty indices array

  2. Enhanced Test Harness with Additional Edge Cases

  Added more comprehensive test scenarios:
  - test_take_reverse - Take elements in reverse order
  - test_take_single_middle - Take single element from middle
  - test_take_random_unsorted - Take with pseudo-random pattern
  - test_take_contiguous_range - Take contiguous range from middle
  - test_take_mixed_repeated - Take with mixed repeated indices
  - test_take_large_indices - Stress test with large number of indices

  3. Added Take Conformance Tests to Array Types

  Successfully added take conformance tests to:
  - PrimitiveArray - Basic numeric arrays
  - BoolArray - Boolean arrays
  - VarBinArray - Variable-length binary arrays
  - VarBinViewArray - Variable-length binary view arrays
  - ChunkedArray - Chunked arrays
  - NullArray - Arrays of all null values
  - ConstantArray - Arrays with constant values
  - ExtensionArray - Extension arrays (fixed dtype sync issue)
  - StructArray - Struct arrays including nested structs
  - ListArray - List arrays with various configurations
  - DecimalArray - D... (continued)

373 of 390 new or added lines in 10 files covered. (95.64%)

8 existing lines in 1 file now uncovered.

45233 of 54693 relevant lines covered (82.7%)

184886.37 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

98.4
/vortex-array/src/compute/conformance/take.rs
1
// SPDX-License-Identifier: Apache-2.0
2
// SPDX-FileCopyrightText: Copyright the Vortex contributors
3

4
use vortex_dtype::Nullability;
5
use vortex_error::VortexUnwrap;
6

7
use crate::arrays::PrimitiveArray;
8
use crate::compute::take;
9
use crate::{Array, Canonical};
10

11
/// Test conformance of the take compute function for an array.
12
///
13
/// This function tests various scenarios including:
14
/// - Taking all elements
15
/// - Taking no elements
16
/// - Taking selective elements
17
/// - Taking with out-of-bounds indices (should panic)
18
/// - Taking with nullable indices
19
/// - Edge cases like empty arrays
20
pub fn test_take_conformance(array: &dyn Array) {
1,876✔
21
    let len = array.len();
1,876✔
22

23
    if len > 0 {
1,876✔
24
        test_take_all(array);
1,876✔
25
        test_take_none(array);
1,876✔
26
        test_take_selective(array);
1,876✔
27
        test_take_first_and_last(array);
1,876✔
28
        test_take_with_nullable_indices(array);
1,876✔
29
        test_take_repeated_indices(array);
1,876✔
30
    }
1,876✔
31

32
    test_empty_indices(array);
1,876✔
33

34
    // Additional edge cases for non-empty arrays
35
    if len > 0 {
1,876✔
36
        test_take_reverse(array);
1,876✔
37
        test_take_single_middle(array);
1,876✔
38
    }
1,876✔
39

40
    if len > 3 {
1,876✔
41
        test_take_random_unsorted(array);
1,520✔
42
        test_take_contiguous_range(array);
1,520✔
43
        test_take_mixed_repeated(array);
1,520✔
44
    }
1,520✔
45

46
    // Test for larger arrays
47
    if len >= 1024 {
1,876✔
48
        test_take_large_indices(array);
38✔
49
    }
1,838✔
50
}
1,876✔
51

52
fn test_take_all(array: &dyn Array) {
1,876✔
53
    let len = array.len();
1,876✔
54
    let indices = PrimitiveArray::from_iter(0..len as u64);
1,876✔
55
    let result = take(array, indices.as_ref()).vortex_unwrap();
1,876✔
56

57
    assert_eq!(result.len(), len);
1,876✔
58
    assert_eq!(result.dtype(), array.dtype());
1,876✔
59

60
    // Verify elements match
61
    if let Ok(orig_canonical) = array.to_canonical() {
1,876✔
62
        if let Ok(result_canonical) = result.to_canonical() {
1,876✔
63
            match (&orig_canonical, &result_canonical) {
1,876✔
64
                (Canonical::Primitive(orig_prim), Canonical::Primitive(result_prim)) => {
1,419✔
65
                    assert_eq!(orig_prim.byte_buffer(), result_prim.byte_buffer());
1,419✔
66
                }
67
                _ => {
68
                    // For non-primitive types, check scalar values
69
                    for i in 0..len {
2,005✔
70
                        assert_eq!(
2,005✔
71
                            array.scalar_at(i).vortex_unwrap(),
2,005✔
72
                            result.scalar_at(i).vortex_unwrap()
2,005✔
73
                        );
74
                    }
75
                }
76
            }
NEW
77
        }
×
NEW
78
    }
×
79
}
1,876✔
80

81
fn test_take_none(array: &dyn Array) {
1,876✔
82
    let indices: PrimitiveArray = PrimitiveArray::from_iter::<[u64; 0]>([]);
1,876✔
83
    let result = take(array, indices.as_ref()).vortex_unwrap();
1,876✔
84

85
    assert_eq!(result.len(), 0);
1,876✔
86
    assert_eq!(result.dtype(), array.dtype());
1,876✔
87
}
1,876✔
88

89
#[allow(clippy::cast_possible_truncation)]
90
fn test_take_selective(array: &dyn Array) {
1,876✔
91
    let len = array.len();
1,876✔
92

93
    // Take every other element
94
    let indices: Vec<u64> = (0..len as u64).step_by(2).collect();
1,876✔
95
    let expected_len = indices.len();
1,876✔
96
    let indices_array = PrimitiveArray::from_iter(indices.clone());
1,876✔
97

98
    let result = take(array, indices_array.as_ref()).vortex_unwrap();
1,876✔
99
    assert_eq!(result.len(), expected_len);
1,876✔
100

101
    // Verify the taken elements
102
    for (result_idx, &original_idx) in indices.iter().enumerate() {
37,427✔
103
        assert_eq!(
37,427✔
104
            array.scalar_at(original_idx as usize).vortex_unwrap(),
37,427✔
105
            result.scalar_at(result_idx).vortex_unwrap()
37,427✔
106
        );
107
    }
108
}
1,876✔
109

110
fn test_take_first_and_last(array: &dyn Array) {
1,876✔
111
    let len = array.len();
1,876✔
112
    let indices = PrimitiveArray::from_iter([0u64, (len - 1) as u64]);
1,876✔
113
    let result = take(array, indices.as_ref()).vortex_unwrap();
1,876✔
114

115
    assert_eq!(result.len(), 2);
1,876✔
116
    assert_eq!(
1,876✔
117
        array.scalar_at(0).vortex_unwrap(),
1,876✔
118
        result.scalar_at(0).vortex_unwrap()
1,876✔
119
    );
120
    assert_eq!(
1,876✔
121
        array.scalar_at(len - 1).vortex_unwrap(),
1,876✔
122
        result.scalar_at(1).vortex_unwrap()
1,876✔
123
    );
124
}
1,876✔
125

126
#[allow(clippy::cast_possible_truncation)]
127
fn test_take_with_nullable_indices(array: &dyn Array) {
1,876✔
128
    let len = array.len();
1,876✔
129

130
    // Create indices with some null values
131
    let indices_vec: Vec<Option<u64>> = if len >= 3 {
1,876✔
132
        vec![Some(0), None, Some((len - 1) as u64)]
1,560✔
133
    } else if len >= 2 {
316✔
134
        vec![Some(0), None]
40✔
135
    } else {
136
        vec![None]
276✔
137
    };
138

139
    let indices = PrimitiveArray::from_option_iter(indices_vec.clone());
1,876✔
140
    let result = take(array, indices.as_ref()).vortex_unwrap();
1,876✔
141

142
    assert_eq!(result.len(), indices_vec.len());
1,876✔
143
    assert_eq!(
1,876✔
144
        result.dtype(),
1,876✔
145
        &array.dtype().with_nullability(Nullability::Nullable)
1,876✔
146
    );
147

148
    // Verify values
149
    for (i, idx_opt) in indices_vec.iter().enumerate() {
5,036✔
150
        match idx_opt {
5,036✔
151
            Some(idx) => {
3,160✔
152
                let expected = array.scalar_at(*idx as usize).vortex_unwrap();
3,160✔
153
                let actual = result.scalar_at(i).vortex_unwrap();
3,160✔
154
                assert_eq!(expected, actual);
3,160✔
155
            }
156
            None => {
157
                assert!(result.scalar_at(i).vortex_unwrap().is_null());
1,876✔
158
            }
159
        }
160
    }
161
}
1,876✔
162

163
fn test_take_repeated_indices(array: &dyn Array) {
1,876✔
164
    if array.is_empty() {
1,876✔
NEW
165
        return;
×
166
    }
1,876✔
167

168
    // Take the first element multiple times
169
    let indices = PrimitiveArray::from_iter([0u64, 0, 0]);
1,876✔
170
    let result = take(array, indices.as_ref()).vortex_unwrap();
1,876✔
171

172
    assert_eq!(result.len(), 3);
1,876✔
173
    let first_elem = array.scalar_at(0).vortex_unwrap();
1,876✔
174
    for i in 0..3 {
7,504✔
175
        assert_eq!(result.scalar_at(i).vortex_unwrap(), first_elem);
5,628✔
176
    }
177
}
1,876✔
178

179
fn test_empty_indices(array: &dyn Array) {
1,876✔
180
    let indices = PrimitiveArray::empty::<u64>(Nullability::NonNullable);
1,876✔
181
    let result = take(array, indices.as_ref()).vortex_unwrap();
1,876✔
182

183
    assert_eq!(result.len(), 0);
1,876✔
184
    assert_eq!(result.dtype(), array.dtype());
1,876✔
185
}
1,876✔
186

187
fn test_take_reverse(array: &dyn Array) {
1,876✔
188
    let len = array.len();
1,876✔
189
    // Take elements in reverse order
190
    let indices = PrimitiveArray::from_iter((0..len as u64).rev());
1,876✔
191
    let result = take(array, indices.as_ref()).vortex_unwrap();
1,876✔
192

193
    assert_eq!(result.len(), len);
1,876✔
194

195
    // Verify elements are in reverse order
196
    for i in 0..len {
73,522✔
197
        assert_eq!(
73,522✔
198
            array.scalar_at(len - 1 - i).vortex_unwrap(),
73,522✔
199
            result.scalar_at(i).vortex_unwrap()
73,522✔
200
        );
201
    }
202
}
1,876✔
203

204
fn test_take_single_middle(array: &dyn Array) {
1,876✔
205
    let len = array.len();
1,876✔
206
    let middle_idx = len / 2;
1,876✔
207

208
    let indices = PrimitiveArray::from_iter([middle_idx as u64]);
1,876✔
209
    let result = take(array, indices.as_ref()).vortex_unwrap();
1,876✔
210

211
    assert_eq!(result.len(), 1);
1,876✔
212
    assert_eq!(
1,876✔
213
        array.scalar_at(middle_idx).vortex_unwrap(),
1,876✔
214
        result.scalar_at(0).vortex_unwrap()
1,876✔
215
    );
216
}
1,876✔
217

218
#[allow(clippy::cast_possible_truncation)]
219
fn test_take_random_unsorted(array: &dyn Array) {
1,520✔
220
    let len = array.len();
1,520✔
221

222
    // Create a pseudo-random but deterministic pattern
223
    let mut indices = Vec::new();
1,520✔
224
    let mut idx = 1u64;
1,520✔
225
    for _ in 0..len.min(10) {
9,955✔
226
        indices.push((idx * 7 + 3) % len as u64);
9,955✔
227
        idx = (idx * 3 + 1) % len as u64;
9,955✔
228
    }
9,955✔
229

230
    let indices_array = PrimitiveArray::from_iter(indices.clone());
1,520✔
231
    let result = take(array, indices_array.as_ref()).vortex_unwrap();
1,520✔
232

233
    assert_eq!(result.len(), indices.len());
1,520✔
234

235
    // Verify elements match
236
    for (i, &idx) in indices.iter().enumerate() {
9,955✔
237
        assert_eq!(
9,955✔
238
            array.scalar_at(idx as usize).vortex_unwrap(),
9,955✔
239
            result.scalar_at(i).vortex_unwrap()
9,955✔
240
        );
241
    }
242
}
1,520✔
243

244
fn test_take_contiguous_range(array: &dyn Array) {
1,520✔
245
    let len = array.len();
1,520✔
246
    let start = len / 4;
1,520✔
247
    let end = len / 2;
1,520✔
248

249
    // Take a contiguous range from the middle
250
    let indices = PrimitiveArray::from_iter(start as u64..end as u64);
1,520✔
251
    let result = take(array, indices.as_ref()).vortex_unwrap();
1,520✔
252

253
    assert_eq!(result.len(), end - start);
1,520✔
254

255
    // Verify elements
256
    for i in 0..(end - start) {
18,143✔
257
        assert_eq!(
18,143✔
258
            array.scalar_at(start + i).vortex_unwrap(),
18,143✔
259
            result.scalar_at(i).vortex_unwrap()
18,143✔
260
        );
261
    }
262
}
1,520✔
263

264
#[allow(clippy::cast_possible_truncation)]
265
fn test_take_mixed_repeated(array: &dyn Array) {
1,520✔
266
    let len = array.len();
1,520✔
267

268
    // Create pattern with some repeated indices
269
    let indices = vec![
1,520✔
270
        0u64,
271
        0,
272
        1,
273
        1,
274
        len as u64 / 2,
1,520✔
275
        len as u64 / 2,
1,520✔
276
        len as u64 / 2,
1,520✔
277
        (len - 1) as u64,
1,520✔
278
    ];
279

280
    let indices_array = PrimitiveArray::from_iter(indices.clone());
1,520✔
281
    let result = take(array, indices_array.as_ref()).vortex_unwrap();
1,520✔
282

283
    assert_eq!(result.len(), indices.len());
1,520✔
284

285
    // Verify elements
286
    for (i, &idx) in indices.iter().enumerate() {
12,160✔
287
        assert_eq!(
12,160✔
288
            array.scalar_at(idx as usize).vortex_unwrap(),
12,160✔
289
            result.scalar_at(i).vortex_unwrap()
12,160✔
290
        );
291
    }
292
}
1,520✔
293

294
#[allow(clippy::cast_possible_truncation)]
295
fn test_take_large_indices(array: &dyn Array) {
38✔
296
    // Test with a large number of indices to stress test performance
297
    let len = array.len();
38✔
298
    let num_indices = 10000.min(len * 3);
38✔
299

300
    // Create many indices with a pattern
301
    let indices: Vec<u64> = (0..num_indices)
38✔
302
        .map(|i| ((i * 17 + 5) % len) as u64)
116,736✔
303
        .collect();
38✔
304

305
    let indices_array = PrimitiveArray::from_iter(indices.clone());
38✔
306
    let result = take(array, indices_array.as_ref()).vortex_unwrap();
38✔
307

308
    assert_eq!(result.len(), num_indices);
38✔
309

310
    // Spot check a few elements
311
    for i in (0..num_indices).step_by(1000) {
152✔
312
        let expected_idx = indices[i] as usize;
152✔
313
        assert_eq!(
152✔
314
            array.scalar_at(expected_idx).vortex_unwrap(),
152✔
315
            result.scalar_at(i).vortex_unwrap()
152✔
316
        );
317
    }
318
}
38✔
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc