52
pyx: a Python-native package registry, now in Beta (astral.sh)
submitted 1 year ago by to c/python@programming.dev
18 comments fedilink hide all child comments
 

Related discussion:

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 year ago (4 children)

Nope, build pipeline is for x86 since that's what we deploy on, and we use Debian in the containers and Ubuntu for the host. We've talked about switching to aarch64, but as of yet that hasn't happened.

And I could be misremembering and it was actually scipy that caused issues, I get the two mixed up. We tend to delay bumping versions, and usually the fix is to bump the patch version or something.

no non-pytjon deps

The thing is, Python is designed as a glue language. You're supposed to use native code to speed up the slow parts, that's the whole point.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 1 year ago (3 children)

    The thing is, Python is designed as a glue language. You're supposed to use native code to speed up the slow parts, that's the whole point.

    That'a news to me. Got a source?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 1 year ago (2 children)

    Here's an old paper from Guido van Rossum about Python as a glue language, which specifically calls out its extensibility via C/C++. From my understanding, the general philosophy of Python is to first make it obviously correct, and then optimize the parts that are too slow w/ native code.

    And that's generally how we use it. The majority of our codebase is in regular Python, with some dependencies (i.e. numpy, scipy, torch) in native code because Python is too slow, and then some larger chunks that are 100% native code because the interface between Python and C++ is too slow for complex simulations. But outside of those corners of our application, we have hundreds of thousands of lines of simple Python code spread across a bunch of microservices. If Python didn't offer that easy native-code extensibility, we'd use a different language.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    The link you posted, while interesting, doesn't substantiate the claim that Python was "designed" to be glue code. It is a nice side effect due to its simple yet powerful syntax and extensibility using CPython.

    That non-python dependencies being a problem is not surprising and I don't think any language has really solved that. C/C++ doesn't have a dependency system, no package or library registry nor manager, no universal build system, and no version manager. Building anything of that language alone is a chore. Rust has to be present on the system and in the right version. D, Zig, Go, and others too.

    If you're on Linux there's a good chance the library you want to use is already compiled for you and you can just download it as part of the Python package installation. Building it on linux is peeking into Pandora's box. Building it on non-linux is opening Pandora's box. The only non FHS Linux I've encountered to solve the dependency problem somewhat is NixOS.

    IMO to complain about non-python dependencies is born in ignorance of other build systems. If they can't solve their own problems, it's a tall order to demand it of Python's packaging system. Python isn't perfect, of course, but the problems we're talking about here aren't python problems, IMO.

    Anti Commercial-AI license

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 year ago

    That paper was written a few years after Python was released, and before 2.0 was a thing. The article says it's already being used that way in industry, and reasons why vs just doing native code from the start (faster dev time and whatnot). The way module loading and the whole standard library works facilitates native extension.

    His stated goals for Python were generally:

    • clean code for the interpreter
    • clean syntax to make intent clear
    • easy to learn and use

    They go so far as to reject changes that significantly increase performance if it complicates the code. Why? The simplest explanation is that if you need the speed, building your own native extension is the way to go. That was the way I've been told to use Python: write in Python, and if it's not fast enough, rewrite the slow parts in C/C++.

    LuaJIT is an example of the opposite approach, where the interpreter is designed to be fast enough that you generally don't need to use native extensions, and they prioritize performance over features, unlike Python. As a result, community contributions are very limited, and many people stick with the official runtime.

    Python wouldn't be very successful without native extensions, so I think it's absolutely fair to say it was an early design decision.

  • permalink
  • fedilink
  • source
  • parent