This is a common misconception! It is true that, of the Smalltalk descendants, Python was not designed for speed like Self or Java; but it was not designed to be slow or difficult to compile. The main technique required to implement a Python JIT is virtualizable objects, which allows the JIT to temporarily desynchronize the contents of registers from the contents of the heap; it's not obvious and wasn't in the first few iterations of PyPy. Additionally, it turns out that tracing the meta-level (see docs or the paper) is a key trick: instead of JIT'ing the Python program, the JIT operates on the interpreter, on the Python VM itself!

Nonsense. You are only "supposed" to do that because if you are using Python and you want performance you have no other choice!

Reminds me of people saying you aren't "supposed" to store binaries in git - yeah only because git is so bad at it! When we were all using SVN people would have said "you aren't supposed to have multiple people editing the same file simultaneously" as if it were a fundamental truth.

And Python isn't "a glue language". It's often used like that but there are many many pure Python projects.

i mean, that's what you're supposed to do. python is a glue language, after all.

You shouldn't blindly trust those numbers. If you actually read the Python code a lot of them use ctypes to just delegate the work to C. It even says that at the bottom of the page.

Seems like they've finally started calling these out - search for "contentious" on this page:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python3-go.html

not exactly: https://benchmarksgame-team.pages.debian.net/benchmarksgame/box-plot-summary-charts.html tl;dr it can be true, but not necessarily. fast python code is on the magnitude of 1/5th the speed of go.

the thing with python is that there is a huge difference between fast and slow code. in go, only really slow code is slow, and in rust even slow code is fast.

Absolute rubbish. Go is typically something like 2-3x slower than "native" languages like C and Rust. Python is typically 50-100x times slower. Go is much much closed to C than it is to Python.

go is also not a fast language. it's about halfway between python and c.

but they all solve different problems so that's ok.

That ignores just how slow it is. It's on the order of 100x slower than "fast" languages like Rust and Go, so doubling in speed... while nice for Python Devs doesn't really make it fast on an absolute scale. They need to double it's speed a few more times at least.

it doubled in speed from 3.9 to now, so it's getting there.

lol I get it, but it's involuntary. I really wish I could like snakes but I don't think it's happening anytime soon.

But looms at his little puppy dog face. :3

Please don't show a close-up image of a snake, or at least add a spoiler! It's irrelevant to the topic, and it just sets some of us on edge for no good reason. If it's small, it's fine, but this... holy shit, now I have a headache. It's one of the reasons I stopped coding in Python altogether.

python is a language explicitly designed to resist any form of proper optimization. It just can't be made fast

I gather that you don't actually use PyPy much. On average, I expect PyPy to deliver over a 3x speedup. The main issue is that people have built upon slow libraries like Numpy that only seem fast because CPython is even slower.

The reason that CPython cannot have a JIT added on is architectural and political. The CPython core team refuses to learn lessons from PyPy.

His website still says he works for Microsoft, though it's possible he hasn't updated it yet.

i'm a fan of ladies with complete test coverage

but i'm ok with those who are a fan of type inference.

More ladies for me

I'm a fan of type infrence languages

Python typing system is the reason of its huge flexibility, but as many other scripting language, it’s also what makes it so hard to optimize.

I have no idea how you can change that without bringing some “JIT-oriented” features inside the Python language itself.

I still hope for the best, but I’m unsure we will be able to see an efficient JIT system inside CPython soon.

PyPy had to rewrite the whole Python implementation in a JIT focused way, and even with that, they are (not uncommon) cases when PyPy is slower.

Microsoft cancelled its support for the Faster CPython project in May this year, as part of a round of layoffs

wtf did they actually axe Guido? I thought he was heavily involved in that.

This is not entirely fair because the comparison point has changed. CPython is getting much faster, too, thanks to the optimizations stages before the JIT runs.

Oh btw there are three choices, not two.

1. stdlib dataclasses

2. pydantic dataclasses

3. SQLModel mixin

How to use Mixins

pydantic underneath (pydantic-base) is written in Rust. fastapi is fast cuz of pydantic. fastapi is extremely popular. Cuz it's right there in the word, fast. No matter how crap the usability is, the word fast will always win.

If fastapi is fast then whatever is not fastapi is slow. And there is no convincing anyone otherwise so lets not even try.

Therefore lets do fast. Cuz we already agreed slow would be bad.

normal dataclasses is not fast and therefore it's bad. If it had a better marketing team this would be a different conversation.

SQLModel combines pydantic and SQLAlchemy.

At first i feel in love with the SQLModel docs. Then realized the eye wateringly beautiful docs are missing vital details, such as how to:

1. create a Base without also creating a Model table

2. overload __tablename__ algo from the awful default cls.__name__.lower()

3. support multiple databases each containing the same named table

#2 is particularly nasty. SQLModel.__new__ implementation consists of multiple metaclasses. So subclasses always inherit that worthless __tablename__ implementation. And SQLAlchemy applies three decorators, so figuring out the right witchcraft to create the Descriptor is near impossible. pydantic doesn't support overriding __tablename__

Then i came along

After days of, lets be honest, hair loss and bouts of heavy drinking, posted the answer here.

Required familiarity with pydantic, sqlalchemy, and SQLModel.

If you have not already read through it, there is a ton of useful information about Python's data model in the user manual; it is my go-to resource when I want to do weird stuff with metaclasses and the like.

Furthermore, you might find it interesting to peruse the C code that is generated by Cython, because it gives you a concrete view of the kinds of steps that Python has to go through from a C perspective to work within its data model. (Cython is a bit faster than Python because it does not have to interpret bytecode, but unless you use special directives it still has to e.g. do general attribute lookups whenever you interact with a Python value, even if the value is an integer, and maintain reference counts.)

Finally, you might also get a lot out of skimming through the CPython bytecode instructions, as this has a lot of interesting details about how the bytecode interpreter works in practice.

Some of those details are portable, particularly the behavior of code objects. Function declarations (def statements) bind code objects to names within a namespace; binding within a class namespace will create methods by calling the declared metaclass, which defaults to the type() builtin type object.

Some other details are not portable. CPython stores code objects on the C heap and allocates generic closures; it supports either a dict from strings to locals, or user-declared slots naming a tuple of locals. PyPy automatically computes slots in all cases and supports the dict as a special case. Threads generally share a single heap per interpreter, so the creation of threads doesn't matter for declaring or instantiating objects; note that the community makes this work by pushing the convention that .__init__() methods should not do computation and instead should merely initialize the locals, akin to similar conventions in C++. That said, Jython threads are Java threads, not OS threads like in CPython or PyPy, so normal assumptions about threading may not hold.

You will have to let go of some of your practices around memory management. Python is memory-safe and garbage-collected by default; while sometimes you'll want to remove names or map keys with del, it usually isn't necessary. Similarly, while there are maybe a half-dozen ways to customize class creation and memory layout, it's almost never actually necessary to use more than one of them at a time. Instead, stick to writing Pythonic code, and let runtimes like PyPy worry about performance. PyPy goes fast by simplifying what happens under the hood; if it exposed guaranteed internal structure then it would be slower.

Don't think it has that info.