(i.e. whatever chaotic vigilante nonsense the CNT/FAI were doing in Catalunya)
Calling it 'chaotic' is a bit absurd considering how well the 3 million+ Anarchists were able to collectively organize and run Catalonia, including public transport, food, weapon production, and universal healthcare (during a war with extremely limited resources), as was well documented in Collectives in the Spanish Revolution by Gaston Leval.
they were literallly refusing to raise a proper army to contrubute to the war effort
The Anarchists quickly formed a voluntary militia of 150,000 men and women (making up 33% of the Republican military forces) with a coordinated bottom-up structure, where the soldiers themselves would elect someone to lead their squad, and all the squad leaders would elect a commander to lead the squad leaders, etc. Every leadership position could be recalled by a vote outside of combat if the people below them felt they were not performing to expectations. They did this to be in-line with anarchist principles, and they likely believed if they became an authoritarian top-down military force, it would introduce the possibility of going the way of the USSR.
This structure was doubted and questioned at the time by a military advisor to Buenaventura Durruti (at the time, the effective leader of the militias):
spoiler to save space
One day Pérez Farràs stated his criticisms to Durruti directly: “You can’t fight like that,” he declared. In reply, Durruti said:
I’ve said it once and I’ll say it again: I’ve been an anarchist my whole life and the fact that I’m responsible for this human collectivity won’t change my convictions. It was as an anarchist that I agreed to carry out the task that the Central Committee of Anti-Fascist Militias entrusted to me.
I don’t believe—and everything happening around us confirms this— that you can run a workers’ militia according to classical military rules. I believe that discipline, coordination, and planning are indispensable, but we shouldn’t define them in the terms of the world that we’re destroying. We have to build on new foundations. My comrades and I are convinced that solidarity is the best incentive for arousing individual responsibility and a willingness to accept discipline as an act of self-discipline.
War has been imposed upon us and this battle will be different than those we’ve fought in Barcelona, but our goal is revolutionary victory. This means defeating the enemy, but also a radical change in men. For that change to occur, man must learn to live and conduct himself as a free man, an apprenticeship that develops his personality and sense of responsibility, his capacity to be master of his own acts. The worker on the job not only transforms the material on which he works, but also transforms himself through that work. The combatant is nothing more than a worker whose tool is a rifle—and he should strive toward the same objective as the worker. One can’t behave like an obedient soldier, but as a conscious man who understands the importance of what he’s doing. I know that it’s not easy to achieve this, but I also know that what can’t be accomplished with reason will not be obtained by force. If we have to sustain our military apparatus with fear, then we won’t have changed anything except the color of the fear. It’s only by freeing itself from fear that society can build itself in freedom.
Source of that quote.
From historical records available, the Anarchists forces were surprisingly effective considering they had the most limited resources of any of the Republican factions due to the Stalinists and liberals withholding weapons and supplies much of the time, as well as preferentially withholding the better equipment for themselves. That happened so much, the Anarchist's were seriously considering robbing the republican bank to acquire the gold needed to finally purchase weapons and supplies on the open market, but decided against it when the Liberal/Stalinist government promised once more that they'd buy supplies for them, but they ultimately did nothing again, screwing the Anarchists for the last time until the war was lost.
For anyone else curious, there's a very detailed 6-part documentary on the Spanish Civil war that's quite excellent, and goes into the military and battles aspect as well.