I had a data scientist colleague who did something like this and they were a data engineer. A friend also got shoved into such a role and was in charge of all the data in the company. They tried to create a new team under him for the same pay and he told them to shove it.
Be strong, you might have the cards in your hand here. Ask for a raise and of you want it, a team. You might say you're done, but trust me, the job is never over. There will always be more.
Good luck.
First of all, i want to say thats a huge task youre taking on and it sounds like youre doing one hell of a job at it! It sounds like you need a pay raise though, especially if you're still taking on some of the responsibilities that you're supposed to have. Another commenter said that it sounds like a data scientist role, although I think its more both data scientist and data engineering given that you have to consolidate data AND run analysis. Not saying that data scientists don't do any data consolidation work, but it sounds like its's a large part of your role right now.
This was what I was looking for, thank you so much!
If it's fun, profit! I know nothing about what you do but the first job that comes to my mind is "data scientist." Here are some links:
It seems that a lot of data science is focused on AI nowadays, but if you stick with Python you can do everything you want. Also using Python instead of Excel is smart since you achieved that, you're an "engineer data scientist." But since you're the only one, you would be the "director of the data science department."
And no, you're not delusional, you did a good job I guess, don't stop.
I'm mostly done so apparently I can do it. What I would want is to get a title and role description etc that matches what I actually do.
Might be time to discuss with your manager about how you don't think you can do this and they should either create a new team for it or outsource it.
So, the fact that l'm using piefed, multiple servers are involved in responding to my request ??
Error 504.
If you request data from a server, chances are that your server needs to request that data from another server - which is 'upstream'.
There can be multiple upstream servers involved in responding to your single request
As far as big name patterns go, it's VSA for me: Vertical Slice Arcitecture. I'm coming from .NET land, so a basic CRUD API to me means...
Haha yeah, the ... vs ..= thing still trips me up sometimes , if Markdown started “optimizing” that, I probably would’ve lost my mind after hours of debugging 😅.
And yep, totally agree on the federation design angle. Push based federation feels like a relic that made sense early on but doesn’t scale well, especially once retries, delivery guarantees, and caching get involved. Pull based + batching could’ve made timing issues way less painful…
Agree with the emphasis on modularity upfront, as it aligns with my YAGNI takeaway, start simple but layer cleanly for future refactors without a full rewrite. In my case, zero users meant no pressure for premature complexity, letting me ship fast and iterate based on actual needs. Experience definitely sharpens that balance. What's one modular pattern you'd recommend for a basic CRUD API?
KISS is life.
This plus "keep it simple stupid" are 2 key things to keep in mind when writing a program. Going back to old code of mine can be painful when I didn't keep these key principals in mind/was too inexperienced.
Planning for the future doesn't mean getting the architecture right the first time, it just means being modular/layered, in such a way that you CAN refactor later, without having to rewrite the whole app from scratch. Which admittedly isn't obvious to solve, without experience under your velt.
0…max_retries
Good thing Rust replaced ... with ..= for inclusive range syntax. Otherwise, the webshit markdown implementation used by Lemmy UI replacing .. with the … ligature would have been confusingly problematic 😉.
And this seems to be yet another case showing that federation was poorly designed, and should have been designed as pull-based (and batchable/packable), instead of endlessly spam-pushing individual messages and hoping for the best.
How about you ask the friend that gave you those files? It's impossible for us to know what these things are through a screenshot.
RunAsTI looks like it is an app. Maybe try running it? (If you’re confident it isn’t malware)
Not at all what you’re asking for but I switched over to Davinci Resolve (free version) and haven’t opened Premiere since.
Great, I'm glad it helps. Good luck! :)
Oh damn, thank you! I didn't know that and it will most probably serve as a starting block to create the exact thing I need. Everything is still confuse is my head, I'm still not sure what I really want, but you clearly helped me. Thank you so much.
If it were me, I would probably create an .html file, add a block, and use the fetch API to fetch the data from the backend and then render/display it via JS/HTML. It’s always possible to size up and add complexity later.
It is totally the kind of things I'd want to do, just an HTML file, some scripts to populate it and call it a day.
I wonder if we can do even simpler today (with a framework that handles the dynamic aspects of the application, not with barebones JS).
You want a stateful application or a template-model-rendering system?
If not, the webbrowsers support fetch API and you can create HTML from that, or set values on the DOM elements.
Personally, I'm not too familiar with JS frontends in particular. I could name some random names, but don't have experience or particular opinions. What I've read, and intuitively agree with, is that many of the most popular frameworks introduce additional complexities and their own state system when the browser nowadays would cover those natively. Newer frameworks that make use of the current browser tech may be better. But I can't name specific names.
I myself, in terms of web frontend frameworks, work with Blazor (dotnet). Upside being direct C#/dotnet integration and development and wide options, downside being the tech complexity of framework between browser and backend and a mixing of HTML and Razor concerns.
If it were me, I would probably create an .html file, add a <script> block, and use the fetch API to fetch the data from the backend and then render/display it via JS/HTML. It's always possible to size up and add complexity later.
Are you saying you could add HTTP endpoints to your services which can then be queried for data? So REST APIs for example?
Yes, this is more or less what I'm thinking of.
Do you want live updates on the UI or is a fetch visualization enough?
Well, depends on what I am doing. Typically, if I'm working on a parser for a new language for instance, i think live update would be a confortable addition. But I guess that any decent JS framework would easily handle processing an input text change event into querying the backend etc.
I didn't know Open telemetry. I'll take a look at it.
Given that I can setup my backend application to deliver the information (a REST API must be enough most of the time I think), what stack would you use to write the frontend asap? What I mean is I don't want to create an entire Angular project and setup multiple files of stuff to have a minimal visualization of my outputs. I've written some Vue.js back then, it was ok. I wonder if we can do even simpler today (with a framework that handles the dynamic aspects of the application, not with barebones JS).
Also, do we still use Bootstrap to have an out-of-the-box CSS with minimal effort? Is there something newer or better?
Thank you so much for your answer
Typically, I could have a program I wrote doing stuff, I may add a little HTTP server feature to it as an entrypoint to its internal and then have a cute webapp to connect to it to visualize what is going on, idk.
Are you saying you could add HTTP endpoints to your services which can then be queried for data? So REST APIs for example?
Do you want live updates on the UI or is a fetch visualization enough?
For simple fetch visualization, creating a simple web app with browser native JavaScript and HTML seems like a fine, simple solution for barebone/hacky visualizations.
If you want live updates, there's a few alternatives. Polling from REST API, long-running streamed responses (http server sent events), or Websocket (continuous connection and communication). Websocket will need the capability on the backend server.
If you're imagining a reporting/monitoring like tool/UI, using OpenTelemetry and one of many existing collect and store and display solutions could be relatively simple setup, with a bit more investment into serving OpenTelemetry data.
There's various technologies and frameworks. You could choose any one, or choose one closer to your tech stack, whatever you use.