Tracking Issue for slice::array_chunks

[lcnr]

The feature gate for the issue is #![feature(array_chunks)].

Also tracks as_(r)chunks(_mut) in #![feature(slice_as_chunks)] -- these were FCPed starting at #74985 (comment) below.

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also uses as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Restrict N to values greater than 0 at compile time instead of using a runtime panic. Make <[T]>::array_* methods fail to compile on 0 len arrays #99471
• Support calls where N == 0 is unreachable due to conditionals Tracking Issue for slice::array_chunks #74985 (comment)
• Adjust documentation (see instructions on rustc-dev-guide)
• Stabilization PR (see instructions on rustc-dev-guide)
  • Stabilize slice_as_chunks library feature #139656

Unresolved Questions

• Iterator vs slices, Does [T]::array_chunks really need to be an iterator? #76354

Implementation history

• implemented in add slice::array_chunks to std #74373
• mut version in Add slice::array_chunks_mut #75021
• as_chunks{_mut} in Add [T]::as_chunks(_mut) #76635
• as_rchunks{_mut} and unchecked versions in Add as_rchunks (and friends) to slices #78818
• const as_(r)chunk in constify slice_as_chunks (unstable) #111453

[joshlf]

Is there any intention of providing the reverse versions of these functions?

[thomcc]

Is there any intention of providing the reverse versions of these functions?

What would that look like?

[joshlf]

Same API, but iterating backwards. I have a use case in which I need to convert either a prefix or a suffix of a slice to an array reference. This API is perfect for the prefix use case, and I'd love to be able to use it for the suffix use case as well.

[newpavlov]

@joshlf
Both slice of arrays and custom iterator type would implement the DoubleEndedIterator, so your use case does not need a separate reverse method.

[joshlf]

That's only true if N evenly divides the length of the slice (see array_chunks docs). Otherwise, the remaining elements are only accessible via the remainder method, and won't be iterated over in the DoubleEndedIterator implementation.

[newpavlov]

Ah, I misunderstood your use case, you want [0, 1, 2, 3, 4, 5, 6] to be split into [4, 5, 6], [1, 2, 3], [0], not into [6], [3, 4, 5], [0, 1, 2]. It indeed requires a separate method.

[joshlf]

Yep, exactly 😃

[thomcc]

Yeah, array_rchunks is definitely worth having.

[sffc]

Also useful would be the inverse operation: [[0, 1, 2], [3, 4, 5]] to [0, 1, 2, 3, 4, 5]. (from &[[T; 3]] to &[T])

[ChayimFriedman2]

Do we need to wait until const_evaluatable_unchecked is stabilized so we can deny N = 0 at compile time to stabilize this feature? Perhaps we can just mention in the documentation that currently it panics but it may be uplifted to a compile time error in the future.

[fzyzcjy]

Hi, is there any alternatives before this is stable? Thanks!

[CryZe]

Hi, is there any alternatives before this is stable? Thanks!

Yeah bytemuck::cast_slice and bytemuck::cast_slice_mut can do mostly the same (with a Pod requirement though)

[fzyzcjy]

Thanks!

[BurntSushi]

I'd like to note that #74985 (comment) for explaining why take the less conservative approach and which real-world code needs to check N wasn't addressed yet.

I personally didn't respond because to me, examples aren't what convinces me to prefer panics. Post-mono errors are what convinces me to prefer panics. (Because I think post-mono errors should be avoided at almost all costs.)

[scottmcm]

I guess my mind got these methods mixed up with split_first_chunk...

I'll just second this point, actually, because I think it's a good one. I used to write .as_chunks().0[0] in examples because there wasn't a .first_chunk().unwrap() so it was by far the easiest way to get an array. But thankfully we have better ways than that horrible snippit now 🙂

the length of the slice being the (mathematically undefined) dividend

A really interesting (but slightly off-topic) thing this makes me think about: Okasaki chapter 9 is about numerical representations, and the equivalence between a way of writing a number and a container of that length. The simplest example is that a linked list is a unary representation of its own length. The book does a neat job of extending that to other things too, like what the container equivalent of a binary number would be -- and how the push/concat implementation on the data structure ends up with the same complexity as the increment/addition operator on a binary number.

So from that perspective, &[T] is a number, and first_chunk is checked_sub on that number, as_chunks is /+% on that number, and as_flattened is *-by-a-constant on it.

[joboet]

A really interesting (but slightly off-topic) thing this makes me think about: Okasaki chapter 9 is about numerical representations, and the equivalence between a way of writing a number and a container of that length. The simplest example is that a linked list is a unary representation of its own length. The book does a neat job of extending that to other things too, like what the container equivalent of a binary number would be -- and how the push/concat implementation on the data structure ends up with the same complexity as the increment/addition operator on a binary number.

So from that perspective, &[T] is a number, and first_chunk is checked_sub on that number, as_chunks is /+% on that number, and as_flattened is *-by-a-constant on it.

This is getting very off-topic, but this equivalence is what I found so incredibly fascinating about lambda calculus. It's taught me that data is defined by how you interact with it – and these interactions are basically nothing but folds.

[Kixunil]

@BurntSushi thanks, just note that I've read somewhere that post-mono errors are preferred over panics. I'm not sure if the stance on this changed yet.

[shepmaster]

I've read somewhere

Perhaps you mean this comment?

it had been asserted and then repeated in this thread that the lang team prefers runtime panics over post-mono errors. That's not the case. We like errors to happen as soon as possible. So we prefer pre-mono errors over post-mono errors when possible, but similarly we prefer post-mono errors over runtime errors when that is possible and practical

[Kixunil]

@shepmaster correct, thanks!

[Jules-Bertholet]

I personally didn't respond because to me, examples aren't what convinces me to prefer panics. Post-mono errors are what convinces me to prefer panics. (Because I think post-mono errors should be avoided at almost all costs.)

I would rather wait for the language features (e.g., pattern types) such that this check could become a proper pre-mono error. The standard library is forever. If we stabilize a sub-par API now, we will be stuck with it for as long as people keep using Rust. There is no reason to rush; people can always use an external crate if they need this feature now.

[Kixunil]

FTR, I thought that post-mono errors can be changed to pre-mono but they can't:

fn foo<const N: usize>(x: &[u8]) {
    bar(x.as_chunks::<N>());
}

foo::<42>(y);

Would become invalid if as_chunks suddenly started requiring foo to add where N != 0 bound.
So it'd be really great if it just forced people to pass in specific value.

[RalfJung]

So it'd be really great if it just forced people to pass in specific value.

That's probably possible for the compiler to enforce though it feels ad-hoc. Cc @BoxyUwU

It also feels like it'd be very annoying. I couldn't write functions as a user that wrap as_chunks and that are themselves generic. I don't think we should do that.

[lcnr]

That's probably possible for the compiler to enforce though it feels ad-hoc.

Very much possible, yeah 😁

[hanna-kruppe]

I would rather wait for the language features (e.g., pattern types) such that this check could become a proper pre-mono error. The standard library is forever. If we stabilize a sub-par API now, we will be stuck with it for as long as people keep using Rust. There is no reason to rush; people can always use an external crate if they need this feature now.

In practice what happens is not “people pull in some well-known crate that does it properly” (a quick search found one crate that provides it but it has exactly one dependent). Instead, they either contort their code to make do with stable standard library functions (example I wrote myself recently) that already exist, or they just implement it themselves with some unsafe (hopefully in a way that’s sound for N = 0). There is a small but nonzero amount of friction smeared across the Rust ecosystem due to the standard library not stabilizing this functionality. It’s not obvious to me that “the standard library is forever” outweighs that. The worst case outcome is that it either gets deprecated in a couple of years and a better way is added, or we’ll consider pre-mono errors better but not better enough to be worth the churn of deprecation. Either way, the cost is very small: it’s just a couple inherent methods that are marginally worse than they could’ve been in a perfect world, nothing with deep or far-reaching repercussions.

It’s not even obvious to me that there will be a clearly better way to have pre-mono errors in future Rust. There are vague ideas for what language features could enable them, but there’s no experience with how well they work in practice. So I don’t consider it a given that T-libs-api will say in five years “obviously the API should do XYZ to get a pre-mono error for N = 0”. Some way to put bounds on the values of const parameters will probably be needed eventually, but using it for preventing the specific error we’re talking about here may not be a good API design tradeoff after all.

[BoxyUwU]

@RalfJung yes I suggest something like that upthread but it seems like libs is not interested in deferring supporting generic callers until we know how things will pan out with enforcing the N>0 requirement in the type system. as you noted it would prevent being able to generically wrap array_chunks callsites which may or may not be desirable tradeoff depending on where you lean in terms of just shipping something useful now even though it might wind up being an unfortunate decisions 5 years down the raod

[its-the-shrimp]

I agree with the sentiment to shrimply postpone this until there's a better way to enforce N != 0, this isn't an important trait, the absence of which creates discord in the ecosystem, it's just a couple utility methods, the current signature of which, shall it be stabilised now, will 100% be seen as an unfortunate, rushed decision a couple years down the road.

[its-the-shrimp]

And I don't think we'll never get mechanisms for enforcing that at compile-time, e.g. changing N: usize to N: NonZeroUsize seems pretty much possible in the very near future, I wouldn't be pessimistic about this

[hanna-kruppe]

Note that even if const N: NonZeroUsize becomes possible next year (Boxy’s estimate upthread), it’ll be very unergonomic to use in some cases. For example, since there’s no NonZero literals, you’ll need to write something like as_chunks::<{ NonZero::new(5).unwrap() }>() instead of just as_chunks::<5>(). Maybe hinting the length of the resulting arrays somewhere else could guide inference but I haven’t seen any discussion of how that would work. Similar problems apply to named constants that otherwise would be a plain usize. So without betting on further (speculative) language features that reduce these burdens, the const N: NonZeroUsize option isn’t obviously better.

[Sympatron]

If as_chunks would be stabilized with specific values only, it would be somewhat ad-hoc, but it would also be compatible with all other proposed solutions in the future without any breaking changes. And although it could not be used in all places where it might be useful, it could at least be used in many common cases and it could quite easily be stabilized in the near future.

This wouldn't be the first case where a subset of a feature is stabilized because it is useful enough on it's own (see const-generics).

[BurntSushi]

I largely agree with @hanna-kruppe here with regards to postponing. Also, just more broadly speaking, the downside here is not significant from my perspective. We're not talking about a subtle footgun that could lead to incorrect answers. We're talking about a slightly worse failure mode for one particular corner case. That on its own isn't enough to delay a useful API like this indefinitely IMO.

[Jules-Bertholet]

In practice what happens is not “people pull in some well-known crate that does it properly” (a quick search found one crate that provides it but it has exactly one dependent). Instead, they either contort their code to make do with stable standard library functions (example I wrote myself recently) that already exist, or they just implement it themselves with some unsafe (hopefully in a way that’s sound for N = 0).

I agree that the present situation isn’t absolutely ideal. But I think the way to address it is something like Niko Matsakis’s “preview crates” idea, or the futures crate. Adding a dependency is far less of an issue if people know it’s blessed by the Rust team. And this feature would be a perfect test case!

[hanna-kruppe]

In this particular case I don’t think that’ll really help. I’m not avoiding depending on a crate for this functionality because of trust, but because these functions are so trivial (ca. five lines of code each) that the complication and overhead of discovering a crate and depending on it is not worth it compared to the alternatives I described above.

A similar challenge applies on the Rust project side: who’s gonna be motivated to voluntarily take care of setting up a new repository and crate under the Rust project, publishing it, watch out for issues and PRs (unlikely in this specific case, but possible in theory), etc. when the code in question is so boring and the status quo is perfectly tolerable?

[BurntSushi]

Yup. Same. I would never depend on a micro-crate providing a function like this, no matter who built it.

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.