When you frame it that way you make it sound like slavery was not the most pressing factor on everyone's mind.

If this was unintentional, go read either speeches from the time or the declarations of succession from the southern states.

Our new government is founded upon . . . its foundations are laid, its corner-stone rests, upon the great truth that the negro is not equal to the white man; that slavery – subordination to the superior race – is his natural and normal condition.

• Alexander H. Stephens, VP

An increasing hostility on the part of the non-slaveholding States to the institution of slavery, has led to a disregard of their obligations, and the laws of the General Government have ceased to effect the objects of the Constitution. The States of Maine, New Hampshire, Vermont, Massachusetts, Connecticut, Rhode Island, New York, Pennsylvania, Illinois, Indiana, Michigan, Wisconsin and Iowa, have enacted laws which either nullify the Acts of Congress or render useless any attempt to execute them. In many of these States the fugitive is discharged from service or labor claimed, and in none of them has the State Government complied with the stipulation made in the Constitution ... For twenty-five years this agitation has been steadily increasing, until it has now secured to its aid the power of the common Government. Observing the forms of the Constitution, a sectional party has found within that Article establishing the Executive Department, the means of subverting the Constitution itself. A geographical line has been drawn across the Union, and all the States north of that line have united in the election of a man to the high office of President of the United States, whose opinions and purposes are hostile to slavery. He is to be entrusted with the administration of the common Government, because he has declared that that “Government cannot endure permanently half slave, half free,” and that the public mind must rest in the belief that slavery is in the course of ultimate extinction.

• South Carolina's declaration of succession

He was not. The confederates said exactly why the started the war in their declarations and it was, in fact, slavery.

Unless your state wanted the right not to have slavery, the Confederacy wouldn't allow that

This is lost cause propaganda. The confederacy split because of slavery first, every thing was a later justification. The first usage of the term "states rights" wasn't until after the war was over.

Someone better go back in time and tell the authors of the Confederate states' Declarations of Causes for Secession:

Georgia (what follows is quite literally the very first sentence of their declaration. Read the entire thing and tell me it wasn't about slavery. They mention it in nearly every sentence):

The people of Georgia having dissolved their political connection with the Government of the United States of America, present to their confederates and the world the causes which have led to the separation. For the last ten years we have had numerous and serious causes of complaint against our non-slave-holding confederate States with reference to the subject of African slavery.

Mississippi

Our position is thoroughly identified with the institution of slavery-- the greatest material interest of the world.

Oops...

South Carolina (while reading this one, I thought they were the only ones so far to follow the assignment -that is, pretend this wasn't about slavery. But even they couldn't help themselves and had to bring up the Fugitive Slave Act. Oops)

The Constitution of the United States, in its fourth Article, provides as follows: "No person held to service or labor in one State, under the laws thereof, escaping into another, shall, in consequence of any law or regulation therein, be discharged from such service or labor, but shall be delivered up, on claim of the party to whom such service or labor may be due." [...] The same article of the Constitution stipulates also for rendition by the several States of fugitives from justice from the other States.

Texas

She was received as a commonwealth holding, maintaining and protecting the institution known as negro slavery-- the servitude of the African to the white race within her limits-- a relation that had existed from the first settlement of her wilderness by the white race, and which her people intended should exist in all future time.

Virginia (emphasis mine)

The people of Virginia, in their ratification of the Constitution of the United States of America, adopted by them in Convention on the twenty-fifth day of June, in the year of our Lord one thousand seven hundred and eighty-eight, having declared that the powers granted under the said Constitution were derived from the people of the United States, and might be resumed whensoever the same should be perverted to their injury and oppression; and the Federal Government, having perverted said powers, not only to the injury of the people of Virginia, but to the oppression of the Southern Slaveholding States.

https://www.battlefields.org/learn/primary-sources/declaration-causes-seceding-states

So go on about how the Civil War wasn't about slavery, dawg

But it was about slavery, it was about whether or not the federal government have the right to ban slavery.