[–] [S] 2 points 2 months ago

Focusing on x86_64 (and linux), can you expand on what’s missing to reach ABI parity?

I haven't look much into it, but perhaps the sret fix mentioned above would fix some issues on x86-64. For the rest, this is mostly bug fixing (we probably don't handle the alignment everywhere) and fuzzing with ABI cafe.

And somewhat relatedly, are you leaning towards publishing an std component built with the backend or not?

I haven't thought much about this, but I guess an idea would be to ship a std for targets not supported by LLVM.

  • source
  • parent
  • context
  • [–] 1 point 2 months ago (1 child)

    Thank you for your detailed reply.

    Focusing on x86_64 (and linux), can you expand on what's missing to reach ABI parity?

    And somewhat relatedly, are you leaning towards publishing an std component built with the backend or not?

  • source
  • parent
  • context
  • [–] [S] 4 points 2 months ago (2 children)

    Would you expect the backend to be fully ready by the end of the year?

    It depends what you mean by "fully ready", but this is unlikely. By "fully ready", do you mean feature-complete, without any known bugs, working on all GCC targets? Even with only x86-64, I don't expect to have all features: for instance, I don't plan on adding support for sanitizers/code coverage/PGO this year. And some stuff is outside my control: for instance, this Rust PR that will allow me to fix some ABI issues (unblocking, among other things, Aarch64) is waiting for review.

    Or are there big/complicated blockers that remain, or LLVM-isms that make full or nearly full parity impossible?

    Nothing complicated remains as far as I know. I don't think LLVM-isms is an issue anymore, so I would expect close to full parity to be possible (expect maybe supporting Thin LTO). The big remaining tasks are:

    • Support for other architectures than x86-64 (and for SIMD on those targets)
    • Debug info
    • ABI parity

    Relatedly, cranelift cg early struggles had to do, in part, with intrinsics support. I remember talk of somehow solving the problem systemically and not having to implement all of them in the backends directly? Where did things end up regarding that, and how does this backend deal with this problem?

    There are Rust intrinsic fallbacks now, but I do not think all intrinsics have a fallback. And I'm not sure this applies to platform intrinsics.

    And finally, using evcxr with :timing enabled, and switching between backends with :codegen_backend, I notice that gcc is the slowest when it comes to compile time. Is that because more IR is given to gcc to generate from, or is gcc/libgccjit itself slower now?

    Compile times were never been a priority. At first, I thought libgccjit might be slower than GCC since it's not used much, but having used it for another project, it does not seem to be the case. My hunch is that since we need some hacks to convert from the lower-level IR MIR to the higher-level GCC IR, it's going to be hard to generate a very good IR in the end. So, we get a MIR that is not super optimized in terms of compile times and make it worse, so the impact on compile times probably lies there.

  • source
  • parent
  • context
  • [–] 3 points 2 months ago (3 children)

    I've been watching the number of failing UI tests shrinking fast with excitement.

    Would you expect the backend to be fully ready by the end of the year? Or are there big/complicated blockers that remain, or LLVM-isms that make full or nearly full parity impossible?

    Relatedly, cranelift cg early struggles had to do, in part, with intrinsics support. I remember talk of somehow solving the problem systemically and not having to implement all of them in the backends directly? Where did things end up regarding that, and how does this backend deal with this problem?

    And finally, using evcxr with :timing enabled, and switching between backends with :codegen_backend, I notice that gcc is the slowest when it comes to compile time. Is that because more IR is given to gcc to generate from, or is gcc/libgccjit itself slower now?

  • source
  • [–] 3 points 2 months ago*

    My wrist pain has gotten worse and worse last winter and spring, so I decided to try letting Claude do much of the writing. On my established code bases it's surprisingly not shit. I do get more done, but I'm also much more tired now that I basically do code review 90% of the time I used to code in.

    It's much worse and needs a lot of handholding for new code bases though.

  • source
  • parent
  • context
  • [–] 7 points 2 months ago

    Regarding the rust-glancer announcement itself, I hope the author is aware that lspmux (formerly ra-multiplex) exists, and zram. If lspmux didn't exist, then yes, editor restarts would have been annoying, but that has been a solved problem for a few years already.

    Also, I'm not sure hitting SSD storage too much is good strategy in general, but maybe that's an overstated concern.

    Otherwise, yes, RA could do better in memory consumption and some other aspects. And yes, the LSP protocol itself is not ideal.

    In any case, introducing new ideas and approaches is always good, especially when there is no "fragmentation" concerns, which I think is the case here.

  • source
  • [–] 4 points 2 months ago

    I haven't looked at the code, so my two cents may be irrelevant, but it sounds like you could define a structure that borrows the vector, keeps a separate vector of indices into borrowed vector, and has a current element index.

    struct SortFacadeIterator<'a, T: Ord> {
        data: &'a Vec<T>,
        indices: Vec<usize>,
        position: usize,
    }
    

    Then in new you would need to sort the original vector but instead of mutating it you would store indices of a resulting sorted elements from the original, i.e. when passed ["b", "a", "c"] you would create index storage of [1, 0, 2]. After that you can iterate both ways, returning an element by index:

    impl<'a, T: Ord> SortFacadeIterator<'a, T> {
        fn current(&self) -> &T {
            self.data[self.indices[self.position]]
        }
    }
    

    I think, maybe sorting a vector could be done by enumerating original vector and sort_by a value, also I'm not sure you need a full ordering, but I don't remember what a sort expects

  • source
  • [–] 15 points 2 months ago (3 children)

    From https://github.com/rust-glancer/rust-glancer :

    This project is being built with heavy use of LLMs. LLMs are used as a tool, not as a brain replacement.

    I do consider the code to be my, and I am spending a lot of time caring about the code quality. So I consider it to be rather readable and maintainable. It might not be the idiomatic example of compiler-adjacent tooling (I don't have that much domain experience), but I am working on improving it as I work on it.

    So if it is slop, then it is my slop, and the best way to help is to tell me what's wrong. This way I will be able to learn something and hopefully make the project better.

    I don't know what to think about it.

  • source
  • [–] 1 point 2 months ago* (last edited 2 months ago)

    You would have to either mutate the original vector or clone it, yes. If you really want to iterate through an immutable Vec with only constant memory usage, then you probably can't get around O(n^2) runtime, but maybe there is some time-memory-tradeoff where a small memory usage can get some time reduction.

  • source
  • parent
  • context
  • [–] [S] 1 point 2 months ago (1 child)

    That's essentially where the code started, where the vector was presorted, then moved to a function to be used for a calculation. I want to have the caller be able to pass a borrow of a collection, for the purposes of this problem right now a fixed size vector, and not have to move the vector into the function. The function needs the sorted data, but doesn't need to clone any of the data, just needs to run a calculation over it. Ideally the final implementation can a) take a borrow to a fixed size vector and b) not clone any of the underlying data in order to run its own calculations. The solution I'm toying with now is a sorted iterator, which shouldn't clone the underlying vector's data and should traverse the fixed size vector in a sorted order. Having an iterator whose next returns sorted items from the fixed size vector would be the perfect solution to this issue.

  • source
  • parent
  • context
  • [–] 4 points 2 months ago (2 children)

    Maybe a heap data structure is what you're looking for. Depending on the usecase you might have to first turn an unsorted Vec into a heap (runtime in O(n), which is still faster than fully sorting it), at that point you can both insert new elements and remove the minimum element within a time in O(log n).

  • source
  • [–] 2 points 2 months ago*

    If you have a backing storage and random access, you can implement any sorting function and simply yield/return the values in order rather than commit them into a collection. For example, you can track the minimum value after one full iteration, then iterate in reverse tracking both the current minimum and the next minimum and yielding each current minimum, then repeat going the other direction, and so on until you're returning only the maximums. In theory, this shouldn't require any internal buffer at all, but does require a total ordering constraint on the type (and is O(n^2) and in practice slow af)

  • source
  • parent
  • context
  • [–] [S] 1 point 2 months ago (1 child)

    One of the things I mentioned was that the iterator is defined from a Vector, making it a fixed size iterator. I'm wondering what constraints I might need to figure out implementing a sortediterator, if that means I need to enforce reverse iteration on it, then I'm OK with that. Im trying to minimize memory in this implementation, but if the tradeoff means higher compute time, that's fine with me. What other options are you referring to in your fourth paragraph?

  • source
  • parent
  • context
  • [–] 3 points 2 months ago* (2 children)

    First, let's start by looking at the Iterator trait. It has one method (well, two, but one relevant one):

    fn next(&mut self) -> Option<Self::Item>
    

    If the last item the iterator returns is your minimum value, you need to run the iterator all the way to the end to find and return that value.

    If you do this, in order to return any other value the iterator returned, you needed to buffer it somewhere. You cannot traverse an iterator in reverse. This means that in order to properly sort an iterator, you need to collect the entire iterator into a buffer to first check whether the last value it returned was the minimum and thus should be the first value in the sorted iterator.

    If you have access to a backing buffer or can iterate in reverse, other options become available. You don't need to buffer again because you can access previous elements which are stored somewhere else.

    Iterators in the traditional sense represent streams of data that may or may not have a backing buffer, though. For example, repeat(1) is an infinite length iterator. Sorting an infinite length iterator would be computationally impossible, of course.

  • source
  • [–] 1 point 2 months ago*

    the whole concept of stable distributions is that the software was used for an extended period of time before tha

    Not quite, the whole concept of these ultra stable distros is that they take a version of the software, and essentially pin, it, cherrypicking mostly, or only security changes as updates. The version of the software that is selected, can be, and usually is from a much newer version of the software overall.

    Because of this, Debian and Ubuntu have an interesting pattern, where each one doesn't universally have older packages than the other. Instead, a new Debian release comes out, it has newer packages, then a new Ubuntu release comes out, and they alternate.

    Anyway, during the process of releasing a new stable distro, the programs are libraries are assessed, and then tested a bit before being included. In theory, if you had enough manpower, and you were fast enough, you could probably pin close to the current latest version of things, and then have those in your stable distro, even if they have only been out for a short period of time.

    The idea of "program version has been released for 2 years, now let's include them into a stable distro", is not how they actually work. Instead, the pinning and then security updates model, deduplicates a lot of work, because now you only assess the security and quality of the programs to be packaged once, instead of continuously every single update.

    It's not about eliminating bugs by using well tested programs. Stable distros are about ensuring the stability of the behavior of the system. The same set of, of predictable bugs, rather than a constantly changing set of them.

    If you use a stable distro as your programming language supply chain, you essentially don't have to deal with security updates in dependent libraries, or worry about supply chain security. Any dynamically linked language is able to do this, but my frustration with Rust is that this is not an option, which is something I really hope changes in the future.

  • source
  • parent
  • context
  • view more: ‹ prev next ›