This does seem like an issue with the library you're using. Your second solution, using RawValue, is likely what I would have gone with, bundled with a self-referential type (wrapping the Pin in another nicer-looking type though). This is assuming I want to pass around a 'static type with the partially-deserialized data. In fact, I've done something like this in the past to pass around raw log data next to that same data in a parsed format (where the parsed data borrows from the raw logs).

Alternatively, I'd have deferred the lifetime problem to someone else (library user for example) since the source data is probably provided at a higher level. This is how the libraries you're using do it from what I can tell. They make the LT the user's problem since the user will know what they want to do with the data.

It’s just a nod to the fact that I didn’t go into this project wanting to deal with zero-copy issues - it was decided for me by the library that I needed. Most of the time I do in fact slap Deserialize on a bunch of static types.

I did in the end, of course, but I encountered the two consequences separately and it wasn’t obvious at first.

If I had chosen to write this code in Go I never would have had to think about any of this and I expect it would have been fine.

Well the article is about zero-copy deserialization in Rust. If you just slap #[derive(Deserialize)] on a bunch of 'static types and let it copy the strings, then you don't have this issue in Rust either.

Also, you'd be using Go, not Rust. That's fine, but not really relevant when you want to do JSON deserialization in Rust, is it?

What a wild conclusion to come to.

It's kind of funny how you didn't pick up on the connection between 3 and 4 😉

I'm tired, boss 🥲

Breaking down what async fn in trait does, it converts it to a fn method() -> impl Future<Output=T>, which breaks further down into fn method() -> Self::__MethodRet where the generated associated type implements Future.

This doesn't work for dyn Trait because of the associated type (and the fact method() needs a dyn-safe self parameter).

Instead, your methods need to return dyn Future in some capacity since that return type doesn't rely on associated types. That's where async_trait comes in. Box<dyn Future<...>> is a dyn-safe boxed future, then it's pinned because async usually generates a self-referential type, and you need to pin it anyway to .poll() the resulting future.

Edit: also, being pedantic, associated types are fine in dyn traits, but you need to specify the type for it (like dyn Blah<Foo=i32>. Even if you could name the return type from an async fn, it'd be different for every impl block, so that's not realistic here.

I believe that it would have been much better if there only were namespaces. Would make vetting and trusting specific "publishers" much easier and safer.

Like the many bevy crates. Some official some not. I would like a @bevy namespace. Or one for tokio and so on.

Also unofficial crates or forks would be transparent and would not have to be renamed except for the namespace ofcourse.

I created tracing-proc-macros-ink because the official tracing developers have been namesquatting tracing-proc-macros for six years. I don't blame them but I am still annoyed. The system is just bad.

dyn compatibility of the trait itself is another matter. In this case, an async method makes a trait not dyn-compatible because of the implicit -> impl Future opaque return type, as documented here.

But OP didn't mention whether dyn is actually needed or not. For me, dyn is almost always a crutch (exceptions exist).

I'm not informed on all the details, but a key difference between the async_trait macro and a native async keyword is that async_trait gives you that boxed, trait object type. IIUC the thinking is native support should not automatically box futures, which implies it shouldn't use dyn either. Using Box and dyn is an easy way to make sure the code works no matter what type of future a method returns. But the trade-off is some runtime overhead from heap allocation (due to Box), and dynamic dispatch (due to dyn).

According to areweasyncyet.rs:

async fn in trait method not stabilized yet

• Workaround is available as an attribute macro: async-trait

How are those namespaces getting on?

I ran into the same issue not so long ago and at least for no_std I had to resort to using the async_trait crate. (The project is no_std but has alloc)

I can't recall the exact error so it might have been due to mixing async and non-async methods in the same trait. I would have to look at it again...

Oh I was in your position like A month ago... You're gonna love dealing with Dyn compatibility and Async. I lost half of my hair because I kept pulling on it and smashing my head against the wall

If I understand what you're asking...

This leaves out some details/specifics out to simplify. But basically:

async fn foo() {}

// ^ this roughly desugars to

fn foo() -> impl Future<()> {}

This meant that you couldn't just have (stable) async methods in traits, not because of async itself, but because you couldn't use impl Trait in return positions in trait methods, in general.

Box<dyn Future> was an unideal workaround (not zero-cost, and other dyn drawbacks). async_trait was a proc macro solution that generated code with that workaround. so Box<dyn Future> was never a desugaring done by the language/compiler.

now that we have (stable) impl Trait in return positions in trait methods, all this dance is not strictly needed anymore, and hasn't been needed for a while.

why is common lisp so barebones? it's literally nothing but macros! WHERE'S THE SYNTAX

I see what you mean. OK, I haven't read the rust language specs, RFCs or whatever, so this is my guess why it was done that way.

Regarding derive: Rust doesn't have inheritance but it does have traits / interfaces. This is both an advantage and a disadvantage. For environment with constrained resources, having structs that dont implement a bunch of things you don't need is an advantage. The downside is that in other cases, you need to know what you want.

For example not every class needs to be comparable or know how to print itself to debug output. But if you do want that, you need to know to implement the right trait. Implementing the same trait the same way all the time however is just boilerplate, hence, the derive macro.

At the same time proc-macros like derive ate extremely powerful since they ingest the syntax tree and spit one back out. It allows you to do crazy stuff at compile time with all the IDE goodness of showing docs, finding symbols, and type checking. IDEs can predictably expand macros. I have yet to find an IDE besides CLion that does the same with C/C++ code.

Process macros in rust can do things like consume files (statically as in inserted by the compiler as opposed to dynamically I'm the function browsing the filesystem), which allows reading and interpreting another language, like python or perl, to generate symbols for those other languages. It is thus possible for example to write a python module completely in Rust. With work, it theoretically is possible to also extend python classes (or symbols from other languages) purely in rust.

As for println! and format!, those macros are in the STL. They don't make it heavier. I think they are macros because rust doesn't (or didn't) support variadic arguments.

Yeah, didn't catch your sarcasm there :D

I was just referring to the fact that they are macros.

I also come from python, actual meta-programming concepts are usually implemented via meta-classes, which I would describe as more complex than rust macros, in the "it takes longer to fully understand" sense.

You could also generate python code and execute it if you don't mind the obvious security implications, but that's only a possibility thanks to it being an interpreted language, while rust macros can provide validations and readable error messages out of the box.

To your post's point in general:

• Python uses exceptions, rust uses errors as values, making rust more predictable.
• Python uses Nullables (None | T) which require hand-written handling, instead of rust's Option<T> which has convenience methods and syntax support (... = maybe?).
• Python has GC by default, and making a function fast requires delegating it to another programming language, meaning you usually would write python for software that needs not be performant.

I have more points if you'd like to understand my position, and am willing to explain if you need.

Using a function is strictly worse than figuring out the formatting at compile time (something Zig also does).

The derives are just shortcuts. You can write everything out long-hand like you would in C++ or Python too if you really want.

Honestly both of these complaints are essentially "why does Rust use macros to make writing code better/easier?".

Yup; I was referring to their implementation being dark magic, depending on calling convention, argument type, argument order and possibly other properties specific to the processor’s instruction set…

“If you think you understand va_*, you don’t.”

You can design a language where you don't need to generate code to accomplish this.

Other people have python scripts generate C, so having on in the same codebase and language is certainly an improvement.

My question isn't why this is necessary in Rust. My question is why Rust was designed such that this was necessary.

Because otherwise the compiler team either also needs to maintain a huge amount libraries or cut corners and move things to runtime that really should happen at compile time.

Someone mentioned elsewhere that this allows for compile-time type safety. I'm still trying to wrap my head around how that works.

printf is a great example. According to the C type system, it takes a string and a variable amount of untyped arguments dependent on the content of the string, but that doesn't actually describe the allowed arguments.
Misusing printf like this printf("%s", 42); will get you a warning, but only because there is a special case for printf in the compiler. If you have your own function that does the same as printf, and you misuse the same way, you will find out by dissecting the core dump.

In rust the format string gets parsed at compile time by a macro, which decides the type of each arguments that can than be checked by the compiler. Imagine printf("%s %d",...) created a function with the signature specialized_printf(char* agr0, int arg1), it would be impossible to pass the wrong types of arguments.

Now that these tools exist people have gone further and made a library that checks SQL queries against the shema of a running database and causes a compile error if it doesn't fit.

For functions that want to accept variadic arguments in C/Cpp

While I agree with your post, I do want to call out that Rust's standard library does use a lot of unstable features and calls compiler intrinsics. Anyone can use the unstable features I believe with just #![feature(...)], but not the intrinsics (not that there's much reason to call the intrinsics directly anyway).

You can design a language where you don't need to generate code to accomplish this.

Depending on what you mean by "generate code", the only language at the level of C or C++ that I can think of that does this is Zig. Zig is weird though because you're still doing what is functionally compile-time reflection, so in a way you're still generating code, just in a different way.

If you're comparing to Python, JS, or even C#, those all come with runtimes that can compile/interpret new code at runtime. None of those languages are comparable here. Rust, C, C++, Zig, etc compile into assembly, and type information, impl information, etc are all lost after compilation (ignoring symbol names or anything tracked as debug info).

If you're specifically referring to Debug, Display, PartialEq, etc then the compiler doesn't do that for you because Rust doesn't assume that those traits are valid for everything.

Unlike Java where new Integer(1) != new Integer(1) or JS where "" == 0, Rust requires you to specify when equality comparisons can be made, and requires you to write out the implementation (or use the derive for a simple, common implementation).

Unlike C# where record class Secret(String Value); will print out the secret into your logs when it inevitably gets logged, Rust requires you to specify when a type can be formatted into a string, and how it should be formatted.

Just because a language does things one way doesn't mean every language ever should do things that same way. If you want it to work like another language you like to use, use the language you like to use instead. Rust language designers made explicit decisions to not be the same as other languages because they wanted to solve problems they had with those languages. Those other languages are still usable though, and many solved the same problems in other ways (C#'s nullable reference types, Python's type hints, TypeScript, C++'s concepts, etc).