DSCI is simple yet super flexible pipeline engine to write CI code on regular programming languages, integrates with Forgejo using web hooks. Intended for small teams hosting Forgejo on single VM VPS and willing to create pipelines on regular programming languages

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 3 months ago (5 children)

Looking at the examples, you’ve just made a brand new GitHub Actions framework. There’s YAML to wrap everything together and a bunch of Python that’s so declarative it might as well be HCL. Do you have an example that’s a bit more than “do what YAML does only in bash?”

  • source
  • hideshow 5 child comments
  • [–] [S] 1 point 3 months ago* (last edited 3 months ago) (4 children)

    Ok, try to do it on GH actions, share states between tasks/jobs for example:

    tasks/task_one/task.py

    #!/usr/bin/python3
    
    update_state({
      'out1' : 'out1 value',
      'out2' : 'out2 value'
    })
    

    tasks/task_two/task.py

    #!/usr/bin/python3
    
    dict = get_state()
    print(dict["out1"])
    print(dict["out2"])
    

    Or share states between jobs:

    jobs/job1/task.py

    #!/usr/bin/python3
    
    update_state({
      'out1' : 'out1 value',
      'out2' : 'out2 value'
    })
    

    jobs/job2/task.py

    #!/usr/bin/python3
    
    dict = config()
    
    print(dict["_dsci_"]["job1"]["out1"])
    print(dict["_dsci_"]["job1"]["out2"])
    
    

    I can't imagine how much boilerplate code (if this ever possible ) one needs to write to achieve that on YAML based pipelines (GH Actions/ etc)

    And don't tell me about jobs artifacts ))

    UPDATE:

    Another good example is to run tasks conditionally, yes using old good if:

    #!/usr/bin/python3
    
    if some_condition(foo, bar): 
        run_task(
           'task1', {
              'foo' : 'foo value',
              'bar' : 'bar value'
           }
        )
    

    The same could be quite awkward in YAML based code

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 3 months ago (3 children)

    This is a boilerplate example. I asked for something more than boilerplate. Give me some reasons why I need an incredibly stateful CI engine.

  • source
  • parent
  • hideshow 3 child comments
  • [–] [S] 2 points 3 months ago* (2 children)

    it's just because I think in real world we have a lot of tasks where state is required or extremely beneficial, some examples on top of my head:

    • creation of virtual machines with dynamic IP addresses
    • creation of bug tracking system tickets with unique ticket IDs
    • looking up in databases where fetched records have unique IDs

    etc

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 3 months ago (1 child)

    I wouldn’t do any of that in a CI-only system. If I did, I’d use tools that exist for those jobs that already allow scripting languages.

    1. Why wouldn’t you use IaC tools for this? All the majors have Python already.
    2. What build or deploy task needs to both create and reference a bug ticket?
    3. What build or deploy task needs to do database lookups? Or possibly what task needs those across multiple, independent stages?
  • source
  • parent
  • hideshow 1 child comment
  • [–] [S] 1 point 3 months ago* (last edited 3 months ago)

    If you use tools gluing them into YAML - you get YAML bloated with time . When you say those tools already having Python - excellent I would like to use those Python libs or SDK directly in my Python code instead of juggling those tools as cli or code blocks inside YAML

    UPDATE: and yeah , re-read again - I guess the most of automation is done today via “CI” pipelines even when those are not meant to be CI only, like you said … anyways the rest I have said stands true for me … don’t bake your code into YAML )

  • source
  • parent